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Executive Summary 

The Virginia Department of Education (VDOE) with the Appalachia 
Educational Laboratory (AEL) designed the Model IV Intervention of 
the Partnership for Achieving Successful Schools Initiative (PA+SS). 
The project was implemented in Petersburg City Schools at the start of 
the 2002-2003 school year. The implementation continued through 
the following school year and should be complete in spring 2005. 

The Model IV Intervention is one of four models that provide techni- 
cal assistance to Virginia school divisions and that are now being 
tested by VDOE under the PA+SS project. The Model IV Intervention 
differs from Models I, II, and III because it is directed from the school 
division central office, includes a central office school improvement 
specialist and is designed around the concept of building school capac- 
ity through collaborative work between the school principals and 
school improvement specialists hired under the PA+SS project to assist 
with the Model IV implementation. The model focuses school im- 
provement planning on specific skills, concepts, and content in each of 
the Petersburg City Schools accredited with warning. A warning 
status indicates the school has not fully met state accreditation stan- 
dards, particularly those related to student achievement. Because all 
Petersburg City Schools are accredited with warning, the central office 
specialist is assigned to work with the central administration for up to 
20 hours each week, although over time this role has expanded to 4 
days per week. The main goal of the intervention is to improve stu- 
dent achievement to levels that ensure full accreditation in compliance 
with the state accountability model. Short-term goals include ensuring 
that (1) data are used to make sound instructional decisions at both the 
individual school and district level, (2) school and district faculty have 
the skills needed to develop a strategic plan for improvement based on 
these data, and (3) structures are in place and used to monitor that the 
strategic plan is implemented with fidelity. 



AEL selected The CNA Corporation (CNAC) to conduct an independ- 
ent but coordinated evaluation of the PA+SS Model IV Intervention 
project in the Petersburg City Schools. CNAC is conducting both 
qualitative and quantitative analyses to review and evaluate available 



vi 




data. This report examines and evaluates the fidelity of the Model IV 
Intervention implementation from year one (the 2002-2003 school 
year) through year two (the 2003-2004 school year), by presenting a 
formative evaluation of the implementation processes and a prelimi- 
nary summative evaluation of student achievement. 

The evaluation study employs a quasi-experimental, matched control 
group design using pretest (pre -Model IV Intervention) and posttest 
(post-Model IV Intervention) analysis of achievement data. For com- 
parative purposes, the pass rate data on the state’s Standards of Learn- 
ing (SOL) exams and high school end-of-course tests were collected 
from all 10 Petersburg City Schools and were matched with a control 
group of schools in Richmond City, yielding a sample size of 20 
schools. Matches were made based on several characteristics, such as 
the size of the school, grade levels, poverty indications, and 2001- 
2002 State Standards of Learning (SOL) test results. The SOL tests 
for Virginia Public Schools are criterion-referenced tests and define 
the commonwealth’s expectations for student learning and achieve- 
ment in Grades 3, 5, and 8. The high school end-of-course exams are 
administered in the core courses, which include English, mathematics, 
science, and history/social science. 

The evaluation also includes collection of qualitative data using fo- 
cused telephone interviews with school improvement specialists, 
school principals, central office leaders, and a local board of education 
member. In addition, each school’s most recent school improvement 
plan was reviewed and evaluated using the VDOE Academic Review 
Indicators, a rubric, to determine whether each school improvement 
plan (SIP) met the criteria specified by the state’s rubric. Measures of 
school climate and capacity for improvement were collected by AEL 
and analyzed by CNAC to explore changes in these variables and their 
relationship to school improvement planning processes. 

The main goals of this report are to document the Model IV Interven- 
tion implementation in Petersburg Public Schools and determine 
whether it is beginning to make a difference in student achievement as 
compared to matched control schools. Findings reveal that, overall, 
the Model IV Intervention was implemented with fidelity and student 
achievement is improving in some grades and subjects. The school 
improvement specialists have made inroads in gaining the trust of 
school leaders. There is evidence that the SIPs are being written, moni- 
tored, and revised by some school improvement committees; school 
climate is improving, and increases in student achievement are begin- 
ning to occur. These goals for school improvement, promoted by the 




Model IV Intervention, are demonstrated by the analyses presented in 
each report section. 

The first section, or chapter, of this report documents the Model IV In- 
tervention. The intended audience for this report also is specified. 

The second section documents the roles of the central office specialist, 
school improvement specialists, principals, and other division leaders 
who are the key facilitators of the PA+SS Model IV Intervention. We 
report on their activity types, level of effort, and working relationships. 
We also draw on their weekly activity reports to summarize the chal- 
lenges and obstacles they encountered. During the first 2 years of the 
intervention, the school improvement specialists worked closely with 
school leaders and instructional personnel to foster adoption of a 
meaningful SIP and the use of data to guide instructional decisions. 
The difficulties they encountered included cultural obstacles, training 
and education deficiencies, and management challenges. The school 
improvement specialist alone cannot be expected to cause, or take 
credit for, progress in school performance: a team effort is required for 
the intervention to succeed. Even then, sustaining the progress could 
be problematic. Results from this interim report suggest that much ad- 
ditional time is needed to build capacity and transfonn a low- 
performing school division into one that is higher performing. 

The third section of this report presents an analysis of the SIPs. Vir- 
ginia’s School-Level Academic Review rubric was used to assess 
whether each of the Petersburg schools’ SIPs met the specified criteria. 
The rubric has three main categories of indicators: (1) Content Plan 
strategies with 13 indicators, (2) Implementation strategies with 7 in- 
dicators, and (3) Results strategies with 8 indicators. The rubric 
guides school improvement committee members when shaping a 
school’s SIP content and implementation strategies and specifying 
measurement of school achievement results. As part of the Model IV 
Intervention, the school improvement committees added jump-start 
strategies that do not appear in the state’s rubric. Although all school 
SIPs are written to reflect the indicators important for school im- 
provement, some SIPs are not documented as well as might be ex- 
pected. An analysis of one school’s SIP required listing caveats 
regarding whether the school fully met or did not meet the rubric’s 
guideline indicators. An evaluation of the SIPs for three other schools 
shows a range of compliance issues as defined by the rubric. Two ad- 
ditional schools also require SIP follow-up to determine whether 
strategies and action plans for change are actually being implemented. 




The fourth section presents analyses of the dimensions of school cli- 
mate and capacity for improvement. To illuminate these dimensions, 
which help chart the course to higher school achievement, AEL used 
two measurement tools — the AEL Continuous School Improvement 
Questionnaire (AEL CSIQ) and the AEL Measure of School Capacity 
for Improvement (AEL MSCI). The AEL CSIQ measures the extent to 
which a school exhibits a culture of continuous improvement. The 
AEL MSCI assesses the degree to which a school possesses the poten- 
tial to become a high-perfonning learning community. Surveys were 
scored and analyzed using paired /-tests for each scale on both the 
AEL CSIQ and the AEL MSCI. Dimensions of school climate and 
capacity for improvement show that school improvement is occurring 
but at a different rate for climate (AEL CSIQ) than for capacity for 
improvement (AEL MSCI). The AEL CSIQ shows that perceived 
changes in school culture and climate are readily apparent when 2002 
data are compared with 2004 data. However, changes in capacity for 
improvement show that professional staffs are less satisfied in 2004 
than they were in 2003. 

Section 5 presents an analysis of the SOL tests for Petersburg schools 
and the Richmond comparison group. These tests cover core subjects 
and are administered in Grades 3, 5, and 8, and as end-of-course ex- 
ams in high school. This analysis presents baseline percentage rates of 
students passing the SOLs during the pre-Model IV Intervention pe- 
riod between 1998 and 2002 compared to the first year of the Model 
IV Intervention (2002-2003 school year). These cross-sectional 
analyses show variability in student improvement, but some actual 
gains at the third-grade level can be cited, as the following examples 
illustrate: 

• five of the seven Petersburg elementary schools show gains in 
the percentages of students passing the English SOL exam. 

• six of the seven Petersburg elementary schools demonstrate 
gains in the percentages of students passing the math and his- 
tory SOL exams. 

• All Petersburg elementary schools show gains in the percent- 
ages of students passing the science SOL exam. 

Appendix 2 contains a series of charts that help to visualize trends in 
the SOL scores. The data used in these charts represent averages of 
the pass rates for each school district and for each SOL test and year 
combination. A least-squares linear trend (for the entire period 1998 
to 2003) is plotted for the Richmond control schools. For the Peters- 
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burg schools, a least-squares linear trend is plotted for the pre- 
intervention period (1998 to 2002) and for the intervention period 
(2002-2003). Selected findings show the following: 

• Third-grade SOL tests: The difference in achievement between 
Petersburg schools and Richmond control schools appears to be 
lessening, with Petersburg schools making larger gains and, 
hence, approximating the achievement of the Richmond con- 
trols. 

• Eighth-grade SOL tests: Math and history pass rates for Peters- 
burg schools were generally lower than the control group dur- 
ing the pre-intervention period; however, notable gains have 
been made and the differences between Petersburg and the 
Richmond control are becoming smaller. 

• High school end-of-course tests: Petersburg pass rates were 
generally higher than the Richmond control pass rates during 
the pre-intervention period. Both high schools (Petersburg and 
the Richmond control) made gains in all subject areas. Peters- 
burg City Schools achievement is not increasing at a greater 
rate than Richmond City, but neither is it losing ground. 

Recommendations 

The following conclusions and recommendations can be made based 
on the data from this fonnative evaluation: 

• The Model IV Intervention innovation of assigning a senior 
school improvement specialist to the central office appears to 
be having some positive effects in the Petersburg Division 
based on interview, survey, and achievement data. Therefore, 
the intervention team should work with the Division to ensure 
that the central office specialist’s responsibilities are picked up, 
either by creating a pennanent position or by assigning these 
responsibilities to another central office staff member who has 
the necessary qualifications. 

• From the interview data, it appears that the role of the school 
instructional specialist is key to program sustainability. 1 Ide- 



1 The school instructional specialist is an employee of Petersburg City Schools and assigned to 
a school on a full-time basis, whereas the school improvement specialist is employed by the 



X 




ally, this person would provide continuity in data expertise and 
be available both to provide instruction in data analysis and to 
do data analysis as needed. Institutionalizing this position, af- 
ter ensuring that the instructional specialists are well grounded 
in data analysis, would go a long way toward addressing the 
problem of sustainability in individual schools. Therefore, the 
school improvement specialists should continue to train these 
instructional specialists during the entire three-year interven- 
tion. 

• Because the role of the parent is critical to student success, all 
principals should be required to include parent representa- 
tive^) on each school improvement committee to help develop 
each school’s SIP. 

• An effort should be made to reinforce and encourage the use of 
the SIP as a tool for the school’s administration to gauge 
school progress toward defined goals. The SIP should be “a 
living document” (a theme echoed throughout the school im- 
provement specialist interviews), and plans should be created 
and monitored at the Division and school levels. This would 
help the central office be more involved in supporting princi- 
pals, building school capacity, and maintaining collaborative 
relationships with school leaders. 



Virginia Department of Education and AEL, and is responsible for implementing Model IV 
Intervention. 




Introduction 



The Virginia Department of Education (VDOE) is interested in deter- 
mining best practices for supporting under-performing or low- 
performing schools in its effort to improve student achievement. 
“Low performing” refers to those schools that do not meet the stan- 
dards established and monitored by the state board of education. Pe- 
tersburg City Schools includes seven elementary schools, two middle 
schools, and one high school. All 10 of these schools have high- 
poverty and high-minority student populations, and all schools exhibit 
low perfonnance. Dr. Jo Lynne DeMary, State Superintendent of Pub- 
lic Instruction, requested that the Appalachia Educational Laboratory 
(AEL) work in partnership with VDOE and the Petersburg City 
Schools to design and test the Partnership for Achieving Successful 
Schools Initiative (PA+SS) Model IV Intervention. The goal was to 
help the schools in Petersburg move toward full accreditation in the 
state’s accountability system. The Virginia state accountability system 
provides pressure to help ensure that schools take change initiatives se- 
riously. 

AEL selected The CNA Corporation (CNAC) to conduct an independ- 
ent but coordinated evaluation of the PA+SS Model IV Intervention 
project in the Petersburg City Schools. Headquartered in Charleston, 
West Virginia, AEL is a not-for-profit corporation that provides re- 
search, product development, and professional development services 
to educators, education publishers, and policymakers. Located in Al- 
exandria, Virginia, CNAC is a private, nonprofit research and evalua- 
tion organization with a 60-year history of studying issues of local and 
national interest and providing assistance to policymakers and practi- 
tioners in solving complex social and educational problems. Together, 
these entities are working to design, implement, and evaluate the 
PA+SS Initiative Model IV Intervention in the Petersburg City School 
Division’s 10 schools. 

Model IV Intervention is one of four models now being tested by 
VDOE that provide technical assistance to Virginia school divisions 
under the PA+SS project. However, each model takes a different ap- 
proach. Model I provides technical assistance by a state team of edu- 
cators who conduct an Academic Review (AR) at the school level. 
The AR team looks for things such as evidence of best practices, uses 
of data, curriculum alignment, and professional development. The 
team reviews test scores, conducts observations in classrooms, and in- 
terviews principals. Areas defined for improvement become “Essen- 
tial Actions” that are fonnulated as recommendations and incorporated 
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into the school improvement plan (SIP) soon after the visit. After the 
initial AR school visit, the team follows up at each school through a 
series of visits to determine whether implementation of the Essential 
Actions was accomplished. The Model II intervention functions in the 
same way as Model I with the exception that Model II technical assis- 
tance requires visits from an external teacher and principal who make 
monthly visits to schools accredited with warning. The Model III in- 
tervention involves all of the above. In addition, an external principal, 
trained in school improvement theory and strategies, is assigned to 
work with the principal and faculty in each school accredited with 
warning for one week per month. 

Unlike the other models, the Model IV Intervention calls for putting 
one school improvement specialist in each school and another school 
improvement specialist in the central office (referred to as the central 
office specialist). Of the four models, only Model IV involves work- 
ing at the school division’s central office. Because the state board of 
education has no legal authority to take over failing schools, Model IV 
Intervention is intended to provide support at both the individual 
school and central office levels. The intervention is aligned with the 
state requirement that a school division have an improvement plan 
when there is little improvement in student achievement. In the envi- 
ronment of standards-based reform in Virginia, all schools in the Pe- 
tersburg City School Division are under pressure to improve. 

Purpose of this report 

This report presents the formative evaluation findings of the Model IV 
Intervention implemented in the Petersburg City School Division dur- 
ing the first two years (school years 2002-2003 and 2003-2004) of the 
three-year divisionwide school improvement project. The report pre- 
sents CNAC’s analysis of the fidelity with which AEL implemented 
the Model IV Intervention. It explores the role of the school improve- 
ment specialists, the collaboration among the division’s central office 
and the schools and school improvement specialists, the completeness 
of individual SIPs, and the changes that have occurred in school cli- 
mate and capacity for improvement. This report also presents a com- 
parative analysis of the percentages of students passing the Standards 
of Learning (SOLs) tests during the four years prior to the Model IV 
school improvement intervention and in the one year for which data 
are available since the Intervention was initiated. 
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Audience for this report 

The primary audiences for this report are the Commonwealth of Vir- 
ginia, which instituted the PA+SS models, AEL, the U.S. Department 
of Education’s Institute of Education Sciences, and practitioners and 
policymakers at large who are interested in school improvement, par- 
ticularly in low-performing schools with high concentrations of poor 
and minority students. 

Model IV Intervention 

During the 2002-2003 school year, VDOE and AEL began imple- 
menting Model IV of the PA+SS project in Petersburg City Schools. 
The implementation continued through the 2003-2004 school year and 
should be complete by spring 2005. The Model IV Intervention pro- 
vides a system of external facilitation to help the Petersburg schools 
improve student achievement. The long-term goal of the intervention 
is to improve student achievement to levels that ensure full accredita- 
tion in compliance with the state accountability model. Short-term, the 
intervention aims to ensure that 

1 . data are used to make sound instructional decisions at both the 
individual school and district levels 

2. school and district faculty and staff have the skills needed to 
develop strategic plans for improvement based on these data 

3. structures are in place and used to monitor that the strategic 
plan is implemented with fidelity 

Model IV focuses school improvement planning in specific skills, con- 
cepts, and content by funding school improvement specialists to serve 
as external facilitators in each of the Petersburg City Schools that is 
accredited with warning. A warning status indicates that the school 
has not fully met state accreditation standards, particularly those re- 
lated to student achievement. To help facilitate school improvement, 
the school improvement specialist is a member of the school im- 
provement committee and has direct input into the development of the 
school improvement plan. The Model IV Intervention also features a 
central office school improvement specialist (central office specialist) 
who serves as an external facilitator at the central-office level to en- 
sure coordination of the individual school improvement plans with the 
division improvement initiatives. Following is a more detailed de- 
scription of the three main features of the Model IV Intervention. 
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School improvement specialists 



School improvement specialists are assigned to all 10 schools to pro- 
vide support by guiding school personnel in assessing needs and de- 
veloping strategic improvement plans. Each of the school-based 
school improvement specialists is an experienced, retired educator 
who has successfully led a similar school in the past. All receive AEL 
training that includes use of data for decision making, school im- 
provement planning, curriculum alignment, monitoring instruction, 
and use of assessment to monitor student achievement. They also re- 
ceive in-depth training in facilitation skills. The school improvement 
specialists meet monthly to reflect on their experiences in the schools 
and to receive additional training as needed. 

The school improvement specialists play a role in assessing readiness 
for change and work with school leaders to build a culture that is sup- 
portive, sustainable, and focused on improving student achievement. 
The school improvement specialists are assigned to work on-site at a 
school at least one day each week. The role of the school improve- 
ment specialist is to 

• support the principal, school leadership team, and teachers as 
they use data to detennine the school’s needs related to im- 
proving student achievement 

• provide consultation, support, and direct service to the princi- 
pal and school faculty as they develop a school improvement 
plan with goals, objectives, and strategies targeted to the areas 
of need found through data analysis 

• provide support to the principal and faculty to implement the 
school improvement plan and monitor the success of the strate- 
gies. 

Central office school improvement specialist 

At present, all schools in the Petersburg City Schools division are ac- 
credited with warning. Therefore, a school improvement specialist, re- 
ferred to as the central office specialist, was originally assigned to 
work with the central administration for up to 20 hours each week. 
Because more intense central office support was required to accom- 
plish the goals of the Intervention, over time the central office special- 
ist role has expanded to four days per week. The role of the division- 
level facilitator is to 
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• coordinate and monitor the activities of the school improve- 
ment specialists at each school 

• provide consultation and support to the central office admini- 
stration to ensure that policies and practices support the work 
of the schools in improving student achievement 

• support the superintendent, central office leadership team, and 
school board as they use data to determine the school district’s 
needs related to improving student achievement 

• support the development of a division improvement plan on 
which individual schools will build their improvement efforts 

• build structures and processes to support central office moni- 
toring of the implementation of the improvement plans 

• link the school division to outside resources, such as the Vir- 
ginia Department of Education, for assistance in the improve- 
ment process. 

The school improvement specialists at the school and central office 
levels build the capacity of the school community to use data to de- 
velop and implement a SIP by providing training in data analysis and 
strategic planning and by modeling the trained behaviors. 

School improvement plan 

A key feature of Model IV Intervention is the SIP. The school im- 
provement specialist assigned to each school has the responsibility for 
supporting the principal and faculty by including jump-start strategies 2 
in the SIP to move the school forward quickly. These strategies in- 
clude the following: 

• aligning the written curriculum with the taught curriculum 



2 

" For more information about the jump-start strategies, see the short-term strategies in Corallo, 
C., & McDonald, D. (2003). What works with low-performing schools. Charleston, WV: 
AEL. 
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• aligning local and classroom assessments with the curriculum 
and continuously monitoring student progress 

• analyzing student achievement data, including disaggregated 
data for groups such as minorities and special education stu- 
dents, to identify the most critical needs for which immediate, 
common achievement goals and strategies may be developed 

• putting structures in place to monitor both delivery of the in- 
structional program and student progress toward meeting the 
achievement goals 

• providing additional learning time for students. 

To maintain the momentum gained by the jump-start strategies, each 
school improvement specialist supports his or her assigned school 
principal and faculty to address organizational and cultural changes 
needed to sustain improved student achievement. Through standard- 
ized surveys of school culture, as well as interaction with faculty, ad- 
ministration, and community, the school improvement specialist helps 
to determine the readiness of the faculty and community beyond the 
school to change practices and use data to plan for change and to foster 
faculty collaboration and distribute leadership functions. School im- 
provement specialists influence and shape these strategies through par- 
ticipation on their assigned school’s improvement committee. 



Target group 

The Petersburg City School Division is an urban school division serv- 
ing 5,400 students from kindergarten through 12th grade (K-12). 
More than 85% of the students in the division qualify for free or re- 
duced-price lunch programs. All seven elementary schools and both 
middle schools are designated as Title I schools. Eight of the 10 
schools are at or slightly above the student achievement benchmark 
used to designate schools as low perfonning. 

Evaluation plan 

The evaluation employs a quasi-experimental, matched comparison 
group design that uses pretest and posttest achievement data analyses. 
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For comparative purposes, data were collected in all 10 Petersburg 
City Schools, along with one-to-one matched control group schools in 
Richmond City, 3 yielding a sample size of 20 schools. Matches were 
made based on several characteristics, such as the size of the school, 
grade levels, poverty indications, and 2001-2002 SOL test results. 

The main goals of this study are to document the Model IV Interven- 
tion in Petersburg Public Schools and determine what difference it 
made in student achievement. The core research questions are as fol- 
lows: 



• Goal 1 : To what extent is Model IV implemented with fidelity? 

• Goal 2: How effective was Model IV in improving student 
achievement? 

To address these two broad goals, the overall study seeks to answer 
four specific research questions: 

1. What has been the impact of the school improvement special- 
ists and central office specialist on school culture, capacity for 
improvement, quality of the school improvement plans, and 
student achievement (as measured by nine-week benchmark 
assessments and standardized tests)? Are there other (unin- 
tended) effects of the Model IV intervention? 

2. What role did school improvement specialists play in guiding 
school leadership? 

3. Which students appear to have made greater achievement gains 
through the intervention? 

4. What is the relationship between the level of intensity of the in- 
tervention and growth in student achievement? 

In general, this formative evaluation report focuses on the following 
issues: 



3 This Model IV Intervention was not compared to Models I-III because schools and divisions 
were assigned to the models based on need; therefore, schools assigned to the other models are 
not comparable to the Petersburg schools. In addition, the intervention and research teams did 
not have access to data from schools using the other models. 
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• the role of the school improvement specialists 

• the merits of the school improvement plans 

• the school culture and capacity for improvement 

• patterns and trends in SOL results for both Petersburg schools 
and the Richmond City comparison group schools. 
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Findings 



Role of the school improvement specialists 

The school improvement specialists and the central office school im- 
provement specialist constitute the key ingredients in the Model IV In- 
tervention. This section reports on their activities and levels of effort. 
Data collected from them and from relevant school personnel help also 
to depict the obstacles and opportunities they encountered. We are 
able to characterize the school improvement specialists’ interactions 
with school and central office personnel to gain a picture of their ac- 
complishments and make suggestions for future improvement. 

This section covers three main sources of data: (1) the school im- 
provement specialists’ monthly contractor reports and invoices, (2) 
their weekly journals, and (3) interviews with the school improvement 
specialists and school leaders with whom they interacted. For the cen- 
tral office specialist, we have weekly field notes for the 2002-2003 
year and interview data for 2003-2004. 

The contractor reports and invoices provide a summary of what the 
school improvement specialists did and their level of effort (number of 
days worked per month). The weekly journals provide insights into 
the intervention and its evolution. The interviews, conducted at the 
end of the second year of the initiative, present a more distilled view of 
the process and one with additional dimensions, as we interviewed 
school personnel — particularly principals — and school improvement 
specialists. Combining these data makes it possible to view the same 
event from several perspectives and to develop a rich understanding of 
the situations faced by these key facilitators of the PA+SS initiative 
and their clients. 

Several common themes emerged from these documents. Most are 
problems the school improvement specialists encountered in their 
work, and others are barriers that made implementation of PA+SS 
Model IV more difficult. These themes include the following: 

• mistrust of school improvement specialists by school personnel 

• lack of leadership at several levels 

• micromanagement by school board officers 

• high turnover of teachers and administrators 
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• need for teacher development 

• discipline and classroom management problems 

• mistrust between schools and central office 

• too many new initiatives and not enough follow through 



We discuss these themes later in this section. First, we describe in 
more detail our data sources. 



The data 

Monthly contractor report and invoice 

In the monthly contractor report and invoice (MCRI), the school im- 
provement specialists recorded the number of days they worked each 
month and briefly described the nature of their contributions. We do 
not have complete records for all of the school improvement special- 
ists, but those we have allow us to detennine the level of effort typical 
of a school improvement specialist. 

Interviews 

In summer and fall 2004, the study team conducted interviews via 
telephone with the following seven school improvement specialists as- 
signed to the schools, 4 the central office specialist, principals at most 
of the Petersburg schools, other administrators if the principal was not 
available, and a small group of central office administrators. These in- 
terviews used a standard set of questions aimed at eliciting information 
about the intervention and its perceived success. The protocols for the 
school improvement specialists and Petersburg school personnel were 
slightly different to account for the different perspectives of the two 
groups. They covered the following topics: 

• school improvement specialists’ perceptions of the schools 

• school staffs perceptions of the school improvement special- 
ists 

• role of the school improvement specialists 



4 



Several school improvement specialists worked with more than one school. 
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• pressing needs of the schools 

• whether and how the PA+SS initiative addressed those needs 

• school improvement specialist’s accomplishments 

• challenges and disappointments 

• lessons learned 

• effectiveness of the SIP process 

• AEL’s performance 

• perceptions of the impact of the program 

School improvement specialist field notes 

School improvement specialists were tasked to keep field notes on 
their activities. They were to document not only what they did, but 
also to reflect on and record what they had learned and their percep- 
tions of their effectiveness. The journals vary in their approach and 
content. Several make compelling reading, with lengthy notes detail- 
ing the ups and downs of the school improvement specialist’s experi- 
ences and the reception he or she received from administrators and 
teachers. Another is a terse list of activities with limited commentary. 
The others fall between these two extremes. Because of the variation 
in style and quantity of information, it is not possible to gain a com- 
prehensive picture of the intervention across the entire division from 
this source. To preserve anonymity, we will not identify individual 
school improvement specialists or their school assignments. 



What school-level school improvement specialists did 

This section covers the school improvement specialists’ activities. We 
construct this picture of what the school improvement specialists did 
primarily from two sources: (1) the monthly compensation reports and 
(2) the interviews. We refer to the weekly journals to supplement 
these sources as needed. We cover the activities of the central office 
school improvement specialist in a separate section. 

The school improvement specialists typically began their assignments 
by attempting to meet with the school administrators. In particular, 
they wanted to get the principal onboard to promote their efforts and to 
make that support visible to others with whom they needed to work. 
They tried to get standing invitations to relevant meetings of school 



11 




personnel, including curriculum meetings. They met regularly with 
the school instructional specialists and with division curriculum spe- 
cialists as the need arose. 

The school improvement specialists worked hard to get school admin- 
istrators to focus on the SIP. They served on school improvement 
teams and committees. Depending on the principal, the school im- 
provement specialists were invited to do “walkarounds” in the class- 
rooms and to observe classroom activities. The purpose of these 
walkarounds, which were sometimes conducted with a formal protocol 
and other times conducted more informally, was to provide the princi- 
pal with feedback on the instructional program. The school improve- 
ment specialists helped school staff analyze students’ SOL 
performance. The school improvement specialists also helped their 
schools prepare for a visit by the AR Team, which consisted of educa- 
tors paid by VDOE to conduct an external review of the schools’ in- 
structional programs. 5 They often worked one-on-one with teachers. 
In his or her own way, each school improvement specialist took on the 
activities they were expected to perform according to the PA+SS 
Model IV plan. 

Table 1 lists the ways in which the school improvement specialists 
contributed. We have listed activities found in the school improve- 
ment specialists’ monthly contractor reports and the activities recorded 
in the interviews according to more general activity categories. 6 Many 
of these directly track to elements of the PA+SS Model IV guidelines. 
Others, however, represent strategies devised by the school improve- 
ment specialists to deal with their individual school environments. 
Ideally, it would be interesting to know how often school improvement 
specialists engaged in each of these activities. Because the school im- 
provement specialists were not systematically asked for a frequency of 
how many times they performed a specific activity, we could not pre- 
sent an accurate count. This list should instead be viewed as showing 
simply the scope of school improvement specialist activities. 



5 Recommendations (Essential Actions) from the AR Team were expected to be incorporated 
into the SIP. 

6 When we compare the school improvement specialist accounts of their activities with the 
lists given by the principals, there are differences. For the most part, the interviews are less 
detailed (as should be expected, given that these were conducted over 30- to 45-minute tele- 
phone sessions). It is possible, however, that in some cases the principals were unaware of the 
broader scope of the school improvement specialists’ activities. 
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Table 1 Categories of School Improvement Specialist Activities 



School improvement specialist Activities 


Brought in teaching resources (e.g., Daily Oral Language materials) 


Advocated for school needs at central office 


Conducted workshops (e.g., workshops for teachers and instructional 
specialists on disaggregating SOL data) 


Met with grade-level groups 


Instructed teachers on how to teach test-taking strategies 


Observed classes 


Helped to set up remediation programs 


Taught data analysis (e.g., workshops on interpreting benchmark test 
data) 


Helped staff to use data one-on-one 


Analyzed data 


Helped to set up nine-week benchmark tests 


Helped to create professional development calendar 


Helped to develop and update SIP 


Worked with school improvement team 


Helped school prepare for Academic Review 


Helped to implement instructional models (e.g., proven instructional 
strategies, new math models, Co-nect program) 


Helped to align curriculum to the SOLs 


Helped to implement instructional monitoring system 



Level of effort and contacts 

Data from the MCRIs indicate that the school improvement specialists 
worked between 1 and 2 days a week per school. They also spent time 
facilitating and/or participating in divisionwide workshops and school 
improvement specialist meetings. This level of involvement is in 
keeping with the PA+SS Model IV guidelines. 

According to our data, the school improvement specialists worked 
closely with principals, other members of the school leadership, and 
school instructional specialists. They usually were viewed as members 
of the school improvement team. They also worked with individual 
teachers and students. Some of the school improvement specialists 
worked with central office personnel, such as the curriculum special- 
ists. At times, they brought in outside resource personnel. Individu- 
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ally and in group meetings, they consulted with the other school im- 
provement specialists and compared notes. 

Accomplishments 

From our interviews, we gained a picture of the school improvement 
specialists’ perceptions of their greatest accomplishments. It is inter- 
esting to note that several of the school improvement specialists 
viewed gaining the trust of the principal as their most important ac- 
complishment. It was evident that without this first step, on which 
much effort was expended, little else could have been done. 

More substantive accomplishments were also noted. Several school 
improvement specialists noted that analyzing data was a major accom- 
plishment, whereas others spoke of helping school personnel analyze 
data. A few raised their perceived accomplishments to the level of 
creating capacity within the school by holding workshops to teach the 
staff to analyze data. One school improvement specialist spoke of 
working with the school improvement team and following through 
with the principal to see that team suggestions were implemented. 
Another mentioned working with individual teachers to raise student 
achievement. 



Themes 



In this section, we summarize our data on the role of the school im- 
provement specialists by drawing out themes from their accounts of 
activities. In many cases, these themes identify impediments that hin- 
dered the school improvement specialists from accomplishing their 
goals. More generally, these can be viewed as stumbling blocks in the 
path of meaningful school reform. For this section, the school im- 
provement specialist journals are our primary source, but we also draw 
on our other sources. 

Mistrust of school improvement specialists 

Although not stated explicitly in all journals, it was evident that the 
school improvement specialists often felt unwelcome (as though they 
were interlopers) at their assigned sites. The mistrust was manifested 
in several ways: 

• the schools’ failure to include the school improvement special- 
ists in relevant meetings 
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• the school improvement specialists’ inability to schedule meet- 
ings with principals and central office administrators 

• school personnel failure to attend scheduled meetings with 
school improvement specialists 

In most cases, the mistrust seems to have waned over time as the 
school improvement specialists developed relationships and provided 
useful assistance and advice to their clients. It is clear, however, that 
the unwelcoming attitude of central office staff and school personnel 
was often painfully slow to dissipate. 

Lack of leadership 

Many of the journals contained entries indicating their author’s per- 
ception of a lack of leadership in the school system hierarchy. We 
found this problem to be documented at all levels, not only in the 
elected school board and at the central office, but also in school-level 
leadership. We touch upon the problem at each of these levels in turn. 

School board hierarchy 

Some of the journals recounted problems stemming from perceived 
micromanagement by the division’s school board. This problem mani- 
fested itself in several ways: 

• a great deal of time spent by school leadership responding to 
school board leaders’ questions 

• ad hoc meetings of school staff called by school board leaders 

• ad hoc decisions to change policies that directly affected school 
programs 

• hectoring of school personnel by school board leaders 

Clearly this problem was beyond the school improvement specialists’ 
scope to influence. It did, however, affect the ability of the school sys- 
tem to do its job, and it redirected resources away from the real pur- 
pose of improving student achievement. 
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Central office hierarchy 



The Petersburg City Schools’ central office consists of the superinten- 
dent and his staff, including an assistant superintendent for instruction. 
Reporting to this assistant superintendent are a director of instruction 
and a coordinator of testing and assessment. The central office em- 
ploys a set of curriculum specialists, each of whom specializes in a 
discipline, such as math, science, or English. 

From time to time, the school improvement specialists interacted di- 
rectly with the central office where they observed the relationships be- 
tween personnel at their own schools and central office personnel. 
Several examples of their observations follow: 

During the 2002-2003 year, the superintendent 7 seemed less than pro- 
active about fostering the PA+SS program. Not until fairly late in the 
process did he support the SIP process. Indeed, he was quoted several 
times as declaring that the principals had other priorities, or “more im- 
portant things” to do. Similar complaints were voiced by some school 
improvement specialists about other members of the central office hi- 
erarchy. In particular, a lack of follow-through seemed pervasive. 

The 2002-2003 superintendent brought in a former colleague to con- 
duct a series of workshops for division principals. Most of the school 
improvement specialists saw this program as competing for principals’ 
time and attention, and they (plus some administrators) felt the work- 
shops were too general and did not directly relate to the problems in 
the Petersburg school system. One school improvement specialist, 
however, noted the superintendent’s high level of involvement at these 
meetings and deemed the sessions “excellent.” 

Hiring decisions, for both the staff and individual school administra- 
tors, were sometimes viewed as being politically correct and comfort- 
able rather than focused on choosing the best person for the job. This 
observation ties into the cultural issues noted elsewhere in this report. 

School improvement specialists observed at least one case in which the 
incompetence of an administrator ultimately led to Petersburg schools’ 



7 A new superintendent was hired for the 2003-2004 school year. 
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failure to win a grant that had been all-but-earmarked for them. In this 
regard, a lack of planning and organization was evident, and the 
PA+SS project personnel were called in too late to save the day. 

School hierarchy 

According to the intervention model, the school improvement special- 
ists were to work directly with the principals. This was necessary in 
order for them to be taken seriously by teachers and other instructional 
staff members. In August 2002, the central office specialist conducted 
a workshop with all the principals to orient them to the Model IV In- 
tervention and explain to them the role of the school improvement spe- 
cialists. In several cases, however, the school improvement specialists 
had trouble making and maintaining contact with their principals. 
Meetings went unscheduled or unattended, school improvement spe- 
cialists were not given the support they needed in faculty meetings, 
and school improvement specialists’ initiatives were not made a prior- 
ity. Stiff-arming and stonewalling are words that come to mind in de- 
scribing the relationship between some of the principals and their 
school improvement specialists. Again, the insular nature of the 
school system culture seems relevant. 

Several school improvement specialists complained that their princi- 
pals would not participate in classroom “walkarounds.” These appear- 
ances by the principals were seen by all Model IV staff as important to 
emphasizing the principals’ roles as instructional leaders. The school 
improvement specialists believed that the presence of principals in the 
classrooms sent strong signals to teachers and students that learning is 
important. When principals failed to show up as scheduled, that also 
sent a message. 

Further, the perceived quality of the principals was spotty. Several 
were deemed inexperienced, out of their depths, or ineffectual. These 
perceptions seem related to the level of engagement between the 
school improvement specialist and the principal. It was reported that 
some central office administrators had also shared their concerns about 
some principals’ lack of leadership skills. 

Finally, some members of the school leadership did not appear in- 
vested in the SIP process. One of the school improvement specialists 
commented, in year two of the intervention, that teachers didn’t have 
copies of the SIP. It’s not surprising that the teachers, lacking such a 
basic tool, didn’t always seem fully committed to the process. 
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An administrator at the central office described a range of responses to 
the school improvement specialists in the schools. In his view, “some- 
times schools used school improvement specialists as clerical help; 
and sometimes the school improvement specialist was given responsi- 
bility for all instruction, and principals walked away. Either school 
improvement specialists influenced principals and instilled confidence 
or school improvement specialists ended up doing the principal’s job 
themselves.” 

These observations describe two extremes — giving school improve- 
ment specialists too little or too much responsibility, both of which are 
undesirable — and a middle path that the intervention aimed for: influ- 
encing and inspiring principals. Thus, the school improvement spe- 
cialists need not only skills in instructional practices but also 
interpersonal skills to create the desired effect. This situation can be 
tricky because the school improvement specialists have no authority 
over school personnel; they can only try to influence behavior indi- 
rectly and through their own examples. 

Mistrust between schools and central office 

Some school improvement specialists described an atmosphere of mis- 
trust between school personnel and the central office . 8 At times, 
school administrators were perceived as viewing central office person- 
nel as antagonists. Other school personnel were similarly inclined. At 
one point, the school instructional specialists felt that their jobs were 
under threat. Teachers indicated that the curriculum resources they 
needed never arrived, and communication, in general, between the 
central office and the schools was spotty. 

At one school, it is documented that an academic review was sched- 
uled for the beginning of January and the principal received only one 
week’s notice. In addition, during that week the principal had to deal 
with a meeting at the central office, a curriculum specialist in the 
school to observe, and a visit from the Central Office Support Team. 



This atmosphere should not be regarded as unique to Petersburg Schools. School 
and central office personnel — and teachers and administrators in general — are of- 
ten at odds, given their different demands and priorities. See, for example, Patrick 
Welsh, “Teachers and Administrators: Different Worlds,” St. Petersburg Times, 27 
Oct 2002. 
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This clustering of incidents indicates a lack of coordination and timely 
communication from the central office. 

There were many cases of last-minute demands put on the principals 
by the central administration. Examples were given of principals be- 
ing pulled out of important AR- and SIP-related meetings to respond 
to central office requests. In other cases, principals were given no- 
notice demands for substantial data compilations. In turn, some of 
these requests from the central office may have originated with the 
school board. 

Central office communications were also lacking about more substan- 
tive matters, such as curriculum changes and requirements. For exam- 
ple, the elementary schools had been cited for 2 years for not teaching 
language arts for at least 2 hours per day. It appears that the schools 
failed to respond either because principals were not held accountable, 
or worse, because no one in the central office decreed that the schools 
should allot the 2 hours per day. 

A plethora of programs 

Judging from the school improvement specialists’ journals, administra- 
tors were reaching far and wide to find solutions to the Petersburg City 
Schools’ low SOL scores. Rather than taking a division- wide ap- 
proach to the curriculum, different programs were instituted at differ- 
ent schools. New programs in math were begun at elementary, middle, 
and high school levels. New reading programs were tried as well, and 
the entire school system was teamed with the Chesapeake school sys- 
tem to adapt programs that had succeeded there. 

Several school improvement specialists believed that too much was 
being thrown at the instructional personnel and called for a more fo- 
cused approach to curriculum initiatives. Given the occasional low 
level of teacher skills to begin with, the number of new initiatives 
seems to have been overly ambitious. This is especially evident when 
combined with the requirement placed on the teachers to analyze indi- 
vidual student shortcomings based on test results and to adapt their 
teaching to these individual requirements. 



Turnover 

The school improvement specialists reported that a “high level” of 
turnover among administrators and teaching personnel exacerbated the 
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problems noted above. During the 2 years covered in this report, turn- 
over took place at all levels, from the school board and superintendent 
down. School improvement specialists remarked on the difficulties 
presented by the turnover. A period of adjustment and learning always 
results when an experienced individual leaves and a new one takes his 
or her place. Such turbulence is particularly difficult in a school sys- 
tem working under an unprecedented level of scrutiny and subject to 
numerous initiatives aimed at improving student test scores. Not only 
did newcomers need to cope with the typical adjustments, they also 
had to get up to speed on the many new programs that were introduced 
in the course of the intervention. 

Back to basics 

Perhaps the most prominent message from the journals is that there 
can be no quick fix. The school system is plagued by such fundamen- 
tal problems that the addition of one school improvement specialist to 
each school, 1 or 2 days a week, cannot provide a complete solution. 
A solution is especially problematic at the higher grade levels, where 
students lack basic skills normally acquired in the lower elementary 
grades. Thus, the problem snowballs as a student advances in grade. 

One prerequisite for learning is discipline, which was clearly lacking 
in some schools, especially at the middle school level. Discipline 
problems preoccupied both school administrators and teachers. As a 
result, students who wanted to learn were neglected, which once again 
caused a snowball situation in which student achievement was the 
casualty. 

The school improvement specialists were of the opinion that some 
teachers suffered from a lack of basic skills. Not only were they un- 
skilled at classroom management, they also were seen to lack the skills 
and expertise to impart subject-matter knowledge to their students. 
Given this perceived low level of teaching competence, the numerous 
curriculum initiatives often exacerbated the problem. Teachers strug- 
gling to do their jobs were hard pressed to learn the ins and outs of 
new approaches to the curriculum or to disaggregate the test data to 
provide focused remediation. 

Central office school improvement specialist 

A defining characteristic of the Model IV Intervention is that a school 
improvement specialist (referred to as the central office specialist) 
works at the division’s central office for about 20 hours per week. 



20 




During the first 2 years of the intervention, two different individuals 
held that position. In the 2002-2003 school year, the first central of- 
fice specialist had trouble making inroads into the central office. Ac- 
cording to her successor, “[The] central office staff was very 
dismissive of [her] — acting as if they simply didn’t think of her rather 
than being openly aggressive.” Thus, there was a perceived passive 
but effective resistance to the central office specialist. 

This perception is borne out by the first central office specialist’s 
weekly journal entries. They chronicle a series of meetings where key 
personnel did not show up, a failure on the part of central office per- 
sonnel to consult with her, and other frustrations in trying to do her 
job. To remedy this, in the second year, AEL’s Model IV project 
manager took on the role of the central office school improvement 
specialist. It was hoped that, with his extensive experience and strong 
connection with the Virginia Department of Education, he would be 
able to take the project further. He was helped in this endeavor by the 
departure of the previous superintendent and the arrival of a new one 
who was much more supportive of the intervention and appreciative of 
the second central office specialist’s contributions. 9 We focus our ac- 
count on this second central office specialist’s activity. 

Central office specialist activities 

The presence and activities of the central office specialist are central to 
this intervention. We are able to gain a sense of the central office spe- 
cialist’s activities from his interview and from the school-level school 
improvement specialist’s journals, because his efforts often directly 
supported them. He held principals’ workshops monthly, which cov- 
ered leadership training and other topics. He also met monthly with 
the school improvement specialists as a group. All reported that they 
enjoyed these meetings and appreciated the chance to compare notes 
among themselves. 

The central office specialist’s activities in the second year may be dif- 
ficult to reproduce in another division because, before taking on the 
role of interim assistant superintendent, he served for a time as acting 
interim superintendent, which gave him direct authority over princi- 
pals that another central office specialist is unlikely to have. We are 



9 So appreciative, in fact, that he appointed the central office specialist as Interim 
Assistant Superintendent. 



21 




unable to draw the line between his activities in the dual roles, so will 
describe them all. Because all Petersburg City Schools are accredited 
with warning, the central office specialist, usually assigned to work 
with the central administration for up to 20 hours each week, expanded 
his role to include a 4-day-per-week commitment. 

His overall goal and greatest accomplishment was “to set up a system 
to hold people accountable, school by school and individual by indi- 
vidual, up to and including the principal.” In this regard, he required 
each principal to hold a monthly data conference with him. Also, each 
school got a visit every month from central office staff and AEL staff. 
In addition, the central office specialist improved communication be- 
tween the central office and the schools by holding a series of meet- 
ings with all professional staff and featuring various specialists. 

The central office specialist tackled the problem of teacher skills by 
putting in place initiatives to train elementary and middle school 
teachers to teach reading and math. Teachers were also trained in 
writing up teacher observations and conducting walkabouts. 

In these ways, the central office specialist addressed the fundamental 
problems of establishing accountability, enhancing basic teacher skills, 
and improving communication between the central office and the 
schools. He continues to struggle, however, with the culture’s resis- 
tance to change and, in some cases, a lack of talent among division 
personnel. 
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Merits of the school improvement plans 

Virginia’s School-Level Academic Review rubric was used to assess 
whether the Petersburg Schools’ SIPs met the specified criteria. The 
rubric provided guidelines to school improvement committee members 
as they shaped their school’s SIP content and implementation strate- 
gies, and specified measurement of school achievement results. The 
three indicator strategies are (1) content plan strategies with 13 indica- 
tors, (2) implementation strategies with 7 indicators, and (3) results 
strategies with 8 indicators. The rubric is located in Appendix A. In 
addition, the improvement committee was asked to add the following 
jump-start strategies to their school’s SIP to support the goal of help- 
ing move the school forward more quickly: 

• Align the written curriculum with the taught curriculum. 

• Align local and classroom assessments with the curriculum and 
continuously monitor student process. 

• Analyze student achievement data, including disaggregated 
data for groups such as minorities and special education stu- 
dents, to identity the most critical needs for which immediate, 
common achievement goals and strategies may be developed. 

• Put structures in place to monitor both delivery of the instruc- 
tional program and student progress toward meeting the 
achievement goals. 

• Provide additional learning time for students. 

The School-Level AR rubric was used to evaluate the 10 SIPs for the 
2003-2004 school year. In addition, recommendations for the four 
schools in which full academic reviews were conducted were exam- 
ined and compared for congruence with the content of those schools’ 
SIPs. Tables 2a through 5b present findings that show whether each 
school’s SIP met the criteria associated with each indicator. A nu- 
merical value of 1 indicates that the SIP met a specified criterion; a 
value of 0 indicates that the SIP did not meet the specified criterion; 
and a value of X indicates that the SIP data were unavailable or, in a 
few cases, X indicates that not enough detail was provided to accu- 
rately evaluate the information. 
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Table 2a. School-level Academic Review: Plan Content Indicators 



Indicator 

numbers 

Plan con- 
tent 


Indicator text 


School 

A 


School 

B 


School 

C 


School 

D 


School 

E 


SIP 1.1 


Basing the three-year school improvement 
plan on the results of previous academic 
reviews, as required by the Standards of 
Accreditation 


1 


1 


1 


1 


1 


SIP 1.2 


Developing the three-year school im- 
provement plan with the assistance of par- 
ents and teachers, as required by the 
Standards of Accreditation 


1 


0 


1 


1 


1 


SIP 1.3 


Using baseline data/measures relevant to 
areas for improvement planning 


1 


1 


1 


1 


1 


SIP 1.4 


Establishing clear goals that relate to stu- 
dent achievement 


1 


1 


1 


1 


1 


SIP 1.5 


Establishing yearly, measurable objectives 
or benchmarks that are linked to goals 


1 


1 


1 


1 


1 


SIP 1.6 


Describing the strategies to be imple- 
mented and the specific action steps to be 
taken to meet each objective 


1 


1 


1 


1 


1 


SIP 1.7 


Including data collection activities at regu- 
lar, logical (not random) intervals through- 
out plan as part of strategies/action steps 


1 


1 


1 


1 


1 


SIP 1.8 


Including a system of monitoring student 
progress at regular, logical (not random) 
intervals 


1 


1 


1 


1 


1 


SIP 1.9 


Selecting achievement indicators that are 
appropriate to goals and objectives 


1 


1 


1 


1 


1 


SIP 1.10 


Identifying sources of evidence that are 
appropriate to strategies/action steps 


1 


1 


1 


1 


1 


SIP 1.11 


Identifying person(s) responsible for im- 
plementing strategies/action steps and col- 
lecting data/evidence 


1 


1 


1 


1 


1 


SIP 1.12 


Establishing timelines over a three-year 
period and linking shorter time frames to 
specific action steps/strategies 


1 


1 


1 


1 


1 


SIP 1.13 


Including all nine components required by 
Section 8VAC20-131-310.G of the Stan- 
dards of Accreditation (Refer to TA 
Document in AR User’s Handbook or SOA 
for list of components) 


1 


1 


1 


1 


1 
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Table 2b. School-level Academic Review: Plan Content Indicators 



Indicator 

numbers 

Plan content 


Indicator text 


School 

F 


School 

G 


School 

H 


School 

I 


School 

J 


SIP 1.1 


Basing the three-year school improvement 
plan on the results of previous academic 
reviews, as required by the Standards of 
Accreditation 


1 


1 


1 


1 


1 


SIP 1.2 


Developing the three-year school im- 
provement plan with the assistance of par- 
ents and teachers, as required by the 
Standards of Accreditation 


1 


1 


1 


1 


1 


SIP 1.3 


Using baseline data/measures relevant to 
areas for improvement planning 


1 


1 


1 


1 


1 


SIP 1.4 


Establishing clear goals that relate to stu- 
dent achievement 


1 


1 


1 


1 


1 


SIP 1.5 


Establishing yearly, measurable objectives 
or benchmarks that are linked to goals 


1 


1 


1 


1 


1 


SIP 1.6 


Describing the strategies to be imple- 
mented and the specific action steps to be 
taken to meet each objective 


1 


1 


1 


1 


1 


SIP 1.7 


Including data collection activities at regu- 
lar, logical (not random) intervals 
throughout plan as part of strategies/action 
steps 


1 


1 


1 


1 


1 


SIP 1.8 


Including a system of monitoring student 
progress at regular, logical (not random) 
intervals 


1 


1 


1 


1 


1 


SIP 1.9 


Selecting achievement indicators that are 
appropriate to goals and objectives 


1 


1 


1 


1 


1 


SIP 1.10 


Identifying sources of evidence that are 
appropriate to strategies/action steps 


1 


1 


1 


1 


1 


SIP 1.11 


Identifying person(s) responsible for im- 
plementing strategies/action steps and 
collecting data/evidence 


1 


1 


1 


1 


1 


SIP 1.12 


Establishing timelines over a three-year 
period and linking shorter time frames to 
specific action steps/strategies 


1 


1 


1 


1 


1 


SIP 1.13 


Including all nine components required by 
Section 8VAC20-131-310.G of the Stan- 
dards of Accreditation (Refer to TA Docu- 
ment in AR User’s Handbook or SOA for 
list of components) 


1 


1 


1 


1 


1 
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Table 3a. School-level Academic Review: Implementation Indicators 



Indicator num- 
bers 

Implementation 


Indicator text 


School 

A 


School 

B 


School 

c 


School 

D 


School 

E 


SIP 2.1 


Focusing implementation on improved stu- 
dent achievement 


1 


1 


1 


1 


X 


SIP 2.2 


Implementing strategies and action steps in 
the manner described in plan 


1 


0 


1 


1 


1 


SIP 2.3 


Documenting implementation of strate- 
gies/action steps 


1 


0 


1 


1 


1 


SIP 2.4 


Collecting and compiling data/evidence of 
the degree to which strategies and action 
steps are implemented as described in the 
plan 


1 


0 


1 


1 


1 


SIP 2.5 


Monitoring the efforts of staff when carry- 
ing out their responsibilities 


1 


0 


1 


1 


1 


SIP 2.6 


Meeting established timelines for complet- 
ing strategies/action steps and collecting 
data/evidence 


1 


0 


1 


1 


X 


SIP 2.7 


Establishing procedures for macro-to-micro 
analysis of data 


1 


1 


1 


1 


X 



Table 3b. School-level Academic Review: Implementation Indicators 



Indicator 

numbers 

Implementation 


Indicator text 


School 

F 


School 

G 


School 

H 


School 

I 


School 

J 


SIP 2.1 


Focusing implementation on improved stu- 
dent achievement 


1 


1 


1 


1 


1 


SIP 2.2 


Implementing strategies and action steps in 
the manner described in plan 


1 


1 


1 


1 


0 


SIP 2.3 


Documenting implementation of strate- 
gies/action steps 


1 


1 


1 


1 


0 


SIP 2.4 


Collecting and compiling data/evidence of 
the degree to which strategies and action 
steps are implemented as described in the 
plan 


1 


1 


1 


1 


1 


SIP 2.5 


Monitoring the efforts of staff when carrying 
out their responsibilities 


1 


1 


1 


1 


1 


SIP 2.6 


Meeting established timelines for complet- 
ing strategies/action steps and collecting 
data/evidence 


1 


1 


1 


1 


X 


SIP 2.7 


Establishing procedures for macro-to-micro 
analysis of data 


1 


1 


1 


1 


1 
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Table 4a. School-level Academic Review: Results Indicators 



Indicator 

numbers 

Results 


Indicator text 


School 

A 


School 

B 


School 

C 


School 

D 


School 

E 


SIP 3.1 


Establishing a system for monitoring 
and adjusting the school’s improve- 
ment plan 


1 


1 


1 


1 


1 


SIP 3.2 


Analyzing data/evidence to deter- 
mine the degree to which strate- 
gies/action steps are implemented as 
intended 


1 


X 


1 


1 


1 


SIP 3.3 


Systematically monitoring student 
achievement at regular intervals 
throughout the year to determine 
effectiveness of improvement initia- 
tives 


1 


1 


1 


1 


1 


SIP 3.4 


Using data to determine the impact 
of improvement initiatives on stu- 
dent learning 


1 


X 


1 


1 


1 


SIP 3.5 


Modifying goals and/or objectives 
based upon the analysis of 
data/evidence 


1 


X 


1 


1 


1 


SIP 3.6 


Modifying less successful strategies 
and adding new strategies, as 
needed, to promote continued im- 
provement 


1 


X 


1 


1 


X 


SIP 3.7 


Reaching established student 
achievement benchmarks and/or 
objectives 


1 


X 


1 


1 


X 


SIP 3.8 


Communicating the status of im- 
plementation and the results to 
stakeholders 


1 


1 


1 


1 


1 
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Table 4b. School-level Academic Review: Results Indicators 



Indicator 

numbers 

Results 


Indicator text 


School 

F 


School 

G 


School 

H 


School 

I 


School 

J 


SIP 3.1 


Establishing a system for monitoring 
and adjusting the school’s improvement 
plan 


1 


1 


1 


1 


1 


SIP 3.2 


Analyzing data/evidence to determine 
the degree to which strategies/action 
steps are implemented as intended 


1 


1 


1 


1 


1 


SIP 3.3 


Systematically monitoring student 
achievement at regular intervals 
throughout the year to determine effec- 
tiveness of improvement initiatives 


1 


1 


1 


1 


1 


SIP 3.4 


Using data to determine the impact of 
improvement initiatives on student 
learning 


1 


1 


1 


1 


1 


SIP 3.5 


Modifying goals and/or objectives based 
upon the analysis of data/evidence 


1 


1 


1 


1 


1 


SIP 3.6 


Modifying less successful strategies and 
adding new strategies, as needed, to 
promote continued improvement 


1 


1 


1 


1 


X 


SIP 3.7 


Reaching established student achieve- 
ment benchmarks and/or objectives 


1 


1 


X 


1 


1 


SIP 3.8 


Communicating the status of implemen- 
tation and the results to stakeholders 


1 


1 


1 


1 


1 
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Table 5a. School Improvement Plan: Jump-Start Strategies 



Indicator 

numbers 

Jump-start 


Indicator text 


School 

A 


School 

B 


School 

C 


School 

D 


School 

E 


JS 1 


Aligning the written curriculum with the 
taught curriculum 


1 


1 


1 


1 


1 


JS 2 


Aligning local and classroom assessments 
with the curriculum and continuously moni- 
toring student progress 


1 


1 


1 


1 


1 


JS 3 


Analyzing student achievement data, includ- 
ing data for groups such as minorities and 
special education students, to identify the 
most critical needs for which immediate, 
common achievement goals and strategies 
may be developed 


1 


0 


1 


1 


1 


JS 4 


Putting structures in place to monitor both 
delivery of the instructional program and 
student progress toward meeting the 
achievement goals 


1 


1 


1 


1 


1 


JS 5 


Providing additional learning time for stu- 
dents 


1 


0 


1 


1 


1 



Table 5b. School Improvement Plan: Jump-Start Strategies 



Indicator 

numbers 

JS 


Indicator Text 


School 

F 


School 

G 


School 

H 


School 

I 


School 

J 


JS 1 


Aligning the written curriculum with the 
taught curriculum 


1 


1 


1 


1 


1 


JS 2 


Aligning local and classroom assessments 
with the curriculum and continuously moni- 
toring student progress 


1 


1 


1 


1 


1 


JS 3 


Analyzing student achievement data, includ- 
ing data for groups such as minorities and 
special education students, to identify the 
most critical needs for which immediate, 
common achievement goals and strategies 
may be developed 


1 


1 


1 


0 


1 


JS 4 


Putting structures in place to monitor both 
delivery of the instructional program and 
student progress toward meeting the 
achievement goals 


1 


1 


1 


1 


1 


JS 5 


Providing additional learning time for stu- 
dents 


1 


1 


1 


0 


1 



As Tables 2a through 4b show, in the majority of cases, each 
school’s SIP met most of the criteria specified by the content plan, 
implementation, and results indicators listed in the rubric. Tables 
5a and 5b show this also was true for the jump-start strategies. For 
a few schools, however, caveats are offered for consideration re- 
garding whether the school met or did not meet indicator criterion. 
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School B 



School B shows compliance with 12 of the 13 plan content indica- 
tors, 2 of the 7 implementation indicators, 3 of the 8 results indica- 
tors, and 3 of the 5 jump-start strategies. Caveats regarding 
whether they fully met compliance criteria follow: 

• SIP 1.1 -Value = 1 

Although the school completed a SIP, it appears that a great 
deal of information is missing. 

• SIP 1.2- Value = 0 
Nothing was checked. 

• SIP 1.3- Value = 1 

The score was awarded on the assumption that these results 
were used to generate a SIP. 

• SIP 1.7- Value = 1 

It is unclear whether these data are being collected. 

• SIP 1.11 -Value = 1 

Individuals have not been identified; however, broad cate- 
gories of individuals have been identified (i.e., teachers). 

• SIP 2.2 - Value = 0 

Although action steps call for improvements, it is unknown 
whether School B followed through on the identified objec- 
tives. 

• SIP 2.3 - Value = 0 

Although action steps call for improvements, it is unknown 
whether School B followed through on the identified objec- 
tives. 

• SIP 2.4 - Value = 0 

Although action steps call for improvements, it is unknown 
whether School B followed through on the identified objec- 
tives. 

• SIP 2.5 - Value = 0 

Although action steps call for improvements, it is unknown 
whether School B followed through on the identified objec- 
tives. 
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• SIP 2.6 - Value = 0 

Although action steps call for improvements, it is unknown 
whether School B followed through on the identified objec- 
tives. 

• SIP 3.2- Value = X 

It does not appear that School B has collected the necessary 
data. 

• SIP 3.4- Value = X 

This SIP calls for the use of data, but there is no evidence 
that the school has collected them. 

• SIP 3.5- Value = X 

This SIP calls for the use of data, but there is no evidence 
that the school has collected them. 

• SIP 3.6- Value = X 

This SIP calls for the use of data, but there is no evidence 
that the school has collected them. 

• SIP 3.7- Value = X 

This SIP calls for the use of such data, but there is no evi- 
dence that the school has collected them. 

• JS 2 - Value = 1 

There is no evidence to document implementation of this 
strategy. 



School E 



School E shows compliance with all of the 13 plan content indica- 
tor areas, 4 of the 7 implementation indicator areas, 6 of the 8 re- 
sults indicator areas, and all jump-start strategy areas. Caveats 
regarding whether they met full compliance are as follows: 

1. SIP 2.1 -Value = X 

The SIP identifies the need to edit plans based on student 
achievement; however, it is unclear whether the data are 
currently being collected. 

2. SIP 3.6- Value = X 

Although there is evidence that strategies are being moni- 
tored, it is not clear what steps will be taken if strategies are 
not successful. 
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3. SIP 3.7- Value = X 

There is not enough infonnation to indicate whether indi- 
vidual student objectives are being met, but there are 
benchmark tests at specific intervals. 

4. JS 1 - Value = 1 

The SIP indicates that School E is complying with this 
strategy; however, there are no data available for documen- 
tation. 

5. JS 3 - Value = 1 

There is evidence that goals and strategies were developed 
for special education students, but there is no evidence that 
goals and strategies were developed for minority students. 

School F 



School F shows compliance with all of the 13 plan content indica- 
tor areas, all implementation indicator areas, all results indicator 
areas, and all jump-start strategy areas. A caveat regarding 
whether they fully met one indicator criterion follows: 

1. SIP 1.11 -Value = 1 

School F has identified broad categories of individuals re- 
sponsible for implementing the SIP. 



School G 



School G shows compliance with all of the 13 plan content indica- 
tor areas, all implementation indicator areas, all results indicator 
areas, and all jump-start strategy areas. Caveats regarding whether 
they met full indicator compliance are as follows: 

1. SIP 2.3 - Value = 1 

Implementation is documented; however, data are incom- 
plete. 

2. SIP 2.4 - Value = 1 

Implementation is documented; however, data are incom- 
plete. 

3. SIP 2.6 - Value = 1 
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Some deadlines are met; however, the majority of action 
steps do not have the data necessary to determine compli- 
ance. 

4. SIP 3.2- Value = 1 

Implementation is documented; however, data are incom- 
plete. 

5. SIP 3.7- Value = 1 

Implementation is documented; however, data are incom- 
plete. 

School H 



School H shows compliance with all of the 13 plan content indica- 
tor areas, the 7 implementation indicator areas, 7 of the 8 results 
indicator areas, and the 5 jump-start strategy areas. Caveats re- 
garding whether they did or did not meet full indicator compliance 
are as follow: 

1. SIP 2.6 - Value = 1 

There is documentation supporting compliance, although 
there are no specific data to back it up. 

2. SIP 3.7- Value = X 

There is not enough infonnation to indicate whether indi- 
vidual student objectives are being met, but there are 
benchmark tests at specific intervals. The AR cites this as 
an area for improvement. 

School J 

School J shows compliance with all of the 13 plan content indica- 
tor areas, 4 of the 7 implementation indicator areas, 7 of the 8 re- 
sults indicator areas, and all jump-start strategy areas. Caveats 
regarding whether they met full compliance follow: 

1. SIP 2.2 - Value = 0 

According to the AR, the strategies and action steps are not 
being implemented as outlined in the SIP. 

2. SIP 2.3 - Value = 0 

The SIP does not include any comments in the October 1 
status column. 
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Summary 

All schools did an adequate job of complying with rubric indica- 
tors in SIP 1 Content of Plan. The difficulty arose when trying to 
determine whether schools successfully incorporated the rubric’s 
indicators for SIP 2 Implementation and SIP 3 Results into their 
plans. Few schools had completed a SIP indicator of progress as 
of October 1, 2004. Given the rudimentary nature of the data, it 
was difficult to determine whether schools’ SIPs are being imple- 
mented. 

For example, all school SIPs are written to reflect inclusion of the 
indicators that are important for school improvement, but some of 
the SIPs are not documented as well as might be expected. An ex- 
amination of School B’s SIP calls attention to a range of caveats 
regarding whether the school fully met compliance criteria. The 
examinations of the SIPs from schools E, F, and G also call atten- 
tion to some caveats regarding compliance (e.g., it was difficult to 
determine whether School G had satisfied SIP 2.3: “Documenting 
implementation of strategies/action steps with the limited data 
available”). Schools H and J both require follow-up to determine 
whether SIP strategies and action plans for change are actually be- 
ing implemented. Moreover, it appears that detailed documenta- 
tion is the exception rather than the rule. 
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School culture and capacity for improvement 



Dimensions of school culture and capacity for improvement are in- 
tricately woven into the fabric of school improvement. They are 
defined as important steps for moving schools toward becoming 
high-perfonnance learning communities. 10 To illuminate the di- 
mensions of school climate and capacity for improvement that help 
chart the course to higher achievement, AEL used two measure- 
ment tools: AEL Continuous School Improvement Questionnaire 
(CSIQ) and AEL Measure of School Capacity for Improvement 
(MSCI). We used the AEL CSIQ to survey a total of 368 profes- 
sional staff in 2002, 319 staff in 2003, and 268 staff in 2004 to de- 
termine their opinions about school climate. We also used the 
AEL MSCI to survey 94 school professionals’ opinions about the 
capacity for school improvement in both 2003 and 2004. 

AEL continuous school improvement questionnaire 

The AEL CSIQ helps the school staff gauge its performance on six 
vital dimensions related to continuous school improvement. The 
AEL CSIQ is designed to focus educators on specific activities and 
characteristics that might be overlooked in a discussion or more 
general analysis. To the extent that the perceptions of the profes- 
sional staff accurately reflect the situations, the results identify 
areas of strength and weakness as the school works toward con- 
tinuous improvement. 

The AEL CSIQ requires the professional staff — principals, teach- 
ers, teachers’ aides, media specialists, librarians, counselors, and 
others who have classroom or advisory contact with students and 
parents — to provide their perceptions of how the school rates on 
several dimensions. In this sense, the AEL CSIQ is a self-report 
inventory. That is, school perfonnance on the AEL CSIQ is a 
function of the combined perceptions of the professional staff. In 



10 Information on the validity and reliability of the AEL CSIQ, and the instill- 
ment’s connection to constructs that are part of high-performing learning com- 
munities may be found in Meehan, M.L., Cowley, K. S., Craig, J. R., Balow, N., 
and Childers, R. D., 2003, AEL Continuous School Improvement Questionnaire: 
User Manual and Technical Report. WV : AEL. 
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general, the smallest unit of analysis is the school, and results ap- 
ply most directly to specific schools. 

The AEL CSIQ includes 60 randomly ordered items that make up 
six scales of 10 items each. It takes approximately 30 minutes to 
complete the survey. The scales cover the following: 

• Learning Culture reflects how well the culture of the 
school encourages learning by all students, staff, and ad- 
ministrators. It reflects the extent to which the school em- 
phasizes learning rather than passive compliance, is a safe 
but exciting place to be, and encourages curiosity and ex- 
ploration. It also indicates the extent to which teachers re- 
ceive opportunities and encouragement to reflect on 
teaching practice, work with others, and experiment with 
new ways of teaching. 

• School, Family, and Community Connections reflect the 
degree to which staff perceive that parents and community 
members are involved in and feel part of the school. This 
includes such activities as infonning parents and commu- 
nity, forming meaningful partnerships, maintaining open 
communication, and honoring and respecting diverse points 
of view. 

• Shared Leadership reflects the extent to which staff view 
leadership as being shared; that is, whether school adminis- 
trators dominate decision making or there are mechanisms 
for involving teachers, students, and parents. It measures 
opportunities for leadership development and the extent of 
open, two-way communication. 

• Shared Goals for Learning assesses the extent to which 
the school has clear, focused goals that are understood by 
all members of the school community. In addition, it re- 
flects whether shared goals affect what is taught and how 
teachers teach, drive decisions about resources, focus on re- 
sults for students, and are developed and “owned” by many 
rather than a few. 

• Purposeful Student Assessment reflects the extent to 
which respondents view student assessment data as mean- 
ingful; use data to guide instructional decisions; and believe 
data are communicated to the greater school community, 
including teachers, parents, students, and the general com- 
munity. 
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• Effective Teaching measures the extent to which teachers’ 
practices align with research on effective teaching. It as- 
sesses whether teachers actively engage students in a vari- 
ety of learning tasks, pose questions that encourage 
reflection and higher order thinking, expect students to 
think critically, and use teaching strategies designed to mo- 
tivate students. 

Each item is scored on a 6-point scale. Response options range 
from 1 (Not present) to 6 ( Present to a high degree). The scores of 
the items within a scale are summed for a total scale raw score that 
ranges from a low of 10 to a high of 60. 

Table 6 shows the descriptive statistics on the six AEL CSIQ 
scales for Petersburg City Schools professional staff who com- 
pleted the instrument in 2002 (N = 368), 2003 (N = 319), and 2004 
( N = 268). Of the elementary schools, the learning culture scale 
mean scores in 2004 for Schools B (49.4), D (50.3), and G (49.6) 
are highest, with School D showing a consistent upward trend on 
this scale for the 3 years surveyed. Schools B, D, and G each show 
a consistently upward trend in mean scores on the shared goals for 
learning in 2004: School B (50.0), School D (51.9), and School G 
(49.7.) Schools B, D, and G each show highest mean scores of all 
elementary schools on the purposeful student assessment scales 
during 2004 (51.0, 50.9, and 49.2, respectively). The effective 
teaching scale also shows a steady upward trend in mean scores for 
the 3 years of interest, with Schools C (50.3) and G (51.6) showing 
the highest mean scores in 2004. Of the elementary schools, 
School E decreased the most, showing higher mean scores in 2003 
than in 2004 on all scales except purposeful student assessment; 
however, the decrease in scores did not sink to the low 2002 levels 
shown prior to the Model IV Intervention. 
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Table 6 Descriptive Statistics for the AEL Continuous School Improvement Questionnaire 



Schools 


Mean 


Median 


Std. Dev. 


2002 


2003 


2004 


2002 


2003 


2004 


2002 


2003 


2004 


School A 


VO 

II 

a 


r-> 

n 

II 

a 


a 

II 

ts> 














Learning culture 


45.8 


46.7 


43.9 


49.0 


46.0 


45.0 


9.3 


7.1 


10.0 


School/family/community connections 


43.3 


44.6 


44.9 


47.0 


47.0 


47.0 


10.1 


8.4 


9.9 


Sharing leadership 


46.3 


42.7 


43.8 


47.0 


45.5 


43.5 


7.8 


10.6 


9.3 


Shared goals for learning 


44.3 


46.9 


47.6 


46.5 


47.0 


49.0 


10.2 


5.6 


8.2 


Purposeful student assessment 


44.7 


44.7 


46.7 


45.0 


43.0 


46.5 


9.1 


6.9 


8.8 


Effective teaching 


47.1 


47.6 


47.3 


49.0 


49.0 


47.0 


8.7 


7.7 


10.1 


School B 


fN 

II 

K 


n=17 


n= 16 














Learning culture 


47.6 


51.6 


49.4 


54.0 


54.0 


49.0 


7.5 


7.3 


5.1 


School/family/community connections 


42.5 


47.8 


49.4 


45.0 


51.0 


49.0 


11.4 


11.9 


5.7 


Sharing leadership 


34.1 


45.8 


43.8 


31.0 


51.0 


48.5 


14.7 


13.6 


11.7 


Shared goals for learning 


41.8 


48.5 


50.0 


41.0 


51.0 


51.0 


12.3 


8.6 


5.5 


Purposeful student assessment 


42.9 


47.8 


51.0 


42.0 


50.5 


51.5 


10.8 


10.6 


5.2 


Effective teaching 


49.3 


52.7 


51.0 


49.0 


55.5 


51.0 


8.0 


7.9 


6.5 


School C 


a 

II 

bJ 

Ki 


n= 25 


n= 16 














Learning culture 


44.8 


46.2 


50.3 


46.5 


46.5 


50.5 


7.8 


4.6 


5.5 


School/ family/ community connections 


39.4 


41.6 


46.4 


40.0 


41.0 


47.0 


9.2 


8.1 


9.5 


Sharing leadership 


39.7 


40.8 


43.9 


40.0 


40.0 


43.5 


9.7 


9.0 


10.4 


Shared goals for learning 


43.2 


44.7 


51.9 


44.0 


46.0 


53.0 


10.4 


7.0 


5.9 


Purposeful student assessment 


40.1 


42.4 


50.9 


41.0 


41.0 


51.0 


9.1 


6.2 


6.9 


Effective teaching 


44.9 


45.4 


50.3 


46.0 


46.0 


48.5 


8.9 


4.9 


5.2 


School D 


"'fr 

II 

a 


n=31 


n= 22 














Learning culture 


42.0 


49.5 


46.6 


44.0 


51.5 


48.0 


9.0 


8.1 


12.5 


School/family/community connections 


40.8 


48.8 


45.8 


43.0 


52.0 


50.0 


9.8 


8.9 


12.3 


Sharing leadership 


37.3 


46.8 


41.7 


37.0 


49.0 


46.5 


14.3 


12.2 


14.1 


Shared goals for learning 


39.8 


48.8 


46.9 


42.0 


50.0 


50.0 


10.7 


8.8 


11.2 


Purposeful student assessment 


41.3 


49.4 


47.4 


42.5 


52.0 


50.5 


10.0 


8.6 


11.3 


Effective teaching 


44.6 


52.5 


48.4 


45.0 


54.0 


52.0 


9.4 


7.0 


11.9 


School E 


II 

a 


II 

a 


n=19 














Learning culture 


39.0 


44.5 


40.8 


39.0 


45.0 


42.5 


8.7 


9.0 


10.4 


School/family/community connections 


37.1 


41.6 


39.3 


37.5 


44.0 


41.5 


10.6 


10.0 


11.9 


Sharing leadership 


32.1 


40.0 


36.2 


33.5 


38.0 


40.0 


11.9 


11.2 


14.2 


Shared goals for learning 


37.1 


42.5 


39.4 


38.5 


42.0 


45.0 


10.6 


9.8 


12.9 


Purposeful student assessment 


39.4 


43.5 


44.9 


42.0 


44.5 


46.0 


11.0 


10.2 


10.8 


Effective teaching 


40.7 


45.4 


44.9 


40.0 


47.5 


47.0 


10.0 


10.6 


11.3 
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Table 6 Descriptive Statistics for the AEL Continuous School Improvement Questionnaire (continued) 



Schools 


Mean 


Median 


Std. Dev. 


2002 


2003 


2004 


2002 


2003 


2004 


2002 


2003 


2004 


School F 


n= 33 


«=38 


71=28 














Learning culture 


44.5 


42.9 


47.3 


46.0 


43.0 


47.5 


8.7 


8.6 


7.9 


School/family/community connections 


40.8 


43.9 


45.1 


40.0 


44.0 


44.0 


9.5 


8.5 


8.2 


Sharing leadership 


37.5 


40.4 


45.7 


37.0 


43.0 


47.0 


10.0 


11.4 


9.1 


Shared goals for learning 


40.9 


41.5 


47.6 


42.0 


41.0 


47.0 


8.4 


9.8 


7.9 


Purposeful student assessment 


40.9 


43.6 


49.1 


41.0 


45.0 


48.0 


10.0 


9.4 


7.5 


Effective teaching 


45.9 


43.8 


47.8 


47.0 


44.5 


47.0 


7.5 


9.6 


8.3 


School G 


a 

II 


O 

M 

II 

a 


n 

II 

a 














Learning culture 


45.5 


50.2 


49.6 


47.0 


51.0 


50.0 


8.7 


7.6 


5.4 


School/family/community connections 


43.8 


50.4 


46.4 


44.5 


53.0 


49.5 


8.5 


8.0 


7.0 


Sharing leadership 


42.5 


50.2 


39.9 


44.0 


51.0 


41.0 


8.4 


7.3 


11.1 


Shared goals for learning 


44.3 


48.8 


49.7 


45.0 


51.5 


50.0 


5.9 


7.6 


4.9 


Purposeful student assessment 


44.3 


48.1 


49.2 


47.0 


50.0 


50.0 


9.1 


9.0 


5.8 


Effective teaching 


44.8 


48.9 


51.6 


45.5 


50.0 


50.0 


8.7 


7.9 


4.5 


School H 


h=27 


H=0 


n= 25 














Learning culture 


40.0 




48.5 


41.0 




51.0 


10.5 




7.6 


School/family/community connections 


34.6 




45.6 


34.0 




44.0 


11.5 




7.5 


Sharing leadership 


35.6 




45.0 


36.0 




48.0 


9.4 




8.6 


Shared goals for learning 


35.4 




47.5 


35.0 




49.0 


10.5 




7.3 


Purposeful student assessment 


34.5 




47.0 


33.0 




48.5 


9.4 




7.9 


Effective teaching 


40.6 




47.4 


41.0 




50.0 


11.3 




9.7 


School I 


a 

II 

SO 


a 

II 

bj 


n= 36 














Learning culture 


46.4 


40.5 


44.8 


49.5 


42.0 


45.0 


8.7 


10.6 


9.5 


School/family/community connections 


40.3 


37.7 


40.9 


42.0 


37.0 


41.5 


11.2 


12.5 


10.4 


Sharing leadership 


41.6 


38.4 


33.3 


43.5 


40.5 


35.5 


10.7 


11.7 


14.9 


Shared goals for learning 


41.6 


40.7 


43.3 


42.0 


41.5 


44.0 


9.0 


10.7 


9.1 


Purposeful student assessment 


42.3 


40.1 


45.4 


44.5 


41.0 


46.0 


10.1 


11.2 


7.9 


Effective teaching 


45.5 


43.0 


45.6 


47.0 


44.0 


47.0 


9.1 


10.9 


8.7 


School J 


n=59 


n= 82 


00 

II 

a 














Learning culture 


38.6 


42.3 


41.5 


38.0 


42.0 


45.0 


11.3 


9.7 


11.2 


School/family/community connections 


33.0 


38.0 


36.5 


32.5 


37.0 


37.0 


11.7 


11.2 


12.8 


Sharing leadership 


31.4 


38.2 


38.4 


31.0 


38.0 


41.0 


14.4 


12.7 


13.0 


Shared goals for learning 


35.1 


40.5 


40.8 


33.5 


41.0 


46.0 


12.0 


10.7 


12.4 


Purposeful student assessment 


34.3 


40.0 


40.7 


32.5 


41.0 


42.0 


12.4 


11.1 


11.8 


Effective teaching 


40.2 


43.1 


41.6 


40.5 


44.0 


43.5 


11.3 


9.8 


11.3 


School K 


n= 0 


n=0 


w=8 
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Table 6 Descriptive Statistics for the AEL Continuous School Improvement Questionnaire (continued) 



Schools 


Mean 


Median 


Std. Dev. 


2002 


2003 


2004 


2002 


2003 


2004 


2002 


2003 


2004 


Learning culture 






46.9 






48.5 






5.5 


School/family/community connections 






39.6 






40.5 






8.1 


Sharing leadership 






41.6 






44.0 






7.6 


Shared goals for learning 






40.8 






39.0 






8.7 


Purposeful student assessment 






46.1 






46.5 






5.1 


Effective teaching 






51.6 






51.0 






3.4 


School L 


n= 0 


n=7 


fl=0 














Learning culture 




43.3 






46.0 






9.8 




School/family/community connections 




33.9 






34.0 






11.6 




Sharing leadership 




44.4 






49.0 






14.3 




Shared goals for learning 




44.3 






47.0 






10.1 




Purposeful student assessment 




33.0 






35.0 






7.3 




Effective teaching 




45.1 






51.0 






14.7 





Tables 7 and 8 show the descriptive statistics for matched pairs for 
the CSIQ surveys and the results of paired t - tests for the CSIQ sur- 
veys, respectively. The difference, or gain, scores were calculated 
on the 64 professional staff at Petersburg schools who completed 
the survey in both 2002 and 2004. We compiled summary descrip- 
tive statistics for each of the six scales on which matched paired t- 
tests were calculated for the base year (2002) and school improve- 
ment year two (2004). Because all 64 staff who completed the 
survey in 2002 and 2004 did not answer every question on each 
scale, the matched paired Z-test perfonned on each of the six scales 
for the 2002 and 2004 data ranged from n = 64 to n = 59. The 
findings show that the paired /-test statistic is significant for five of 
the six CSIQ scales: learning culture (n = 60), school/ fam- 
ily/community connections (n = 64), shared goals for learning (n = 
59), purposeful student assessment (n = 61), and effective teaching 
(n = 62), which indicates an increase in the average scale scores 
from 2002 to 2004. 
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Table 7 Descriptive Statistics for Matched Pairs for CSIQ Survey of School Climate 





Mean 


Std. Dev. 




2004 


2002 


2004 


Learning culture 


44.6 


47.7 


8.8 


9.2 


School/family/community connections 


40.3 


44.0 


10.6 


10.9 


Sharing leadership 


40.3 


42.5 


11.9 


12.9 


Shared goals for learning 


42.2 


48.4 


10.1 


9.6 


Purposeful student assessment 


41.5 


48.7 


10.9 


9.0 


Effective teaching 


46.5 


49.0 


8.9 


8.9 



Table 8 Results of Paired t-Tests and Cohen’s d Estimates of Effect Size for CSIQ Survey of School 
Climate 





Paired Differences 


Test Statis- 
tic 


Two-tailed 

Significance 


Effect 

Size 

(d) 


Mean 


Std. Dev. 


Learning culture 


-3.2 


9.0 


-2.7 


<.01 


0.35 


School/family/community 

connections 


-3.7 


9.8 


-3.0 


<.01 


0.37 


Sharing leadership 


-2.3 


12.5 


-1.4 


0.17 




Shared goals for learning 


-6.2 


8.0 


-6.0 


<.01 


0.78 


Purposeful student as- 
sessment 


-7.1 


9.5 


-5.9 


<.01 


0.75 


Effective teaching 


-2.5 


8.9 


-2.2 


0.03 


0.28 



Note: The mean is based on the 2002-2004 value, so a negative mean indicates an increase in subscale 

score. Significant values indicate that the mean is significantly different from zero and, in this 
case, that means a significant increase in subscale score. 



AEL measure of school capacity for improvement 

The 64-item MSCI is designed to assess the degree to which 
schools possess the potential to become high-performing learning 
communities. The survey was developed in response to the pau- 
city of definition, operationalization, and assessment of school ca- 
pacity in the education research environment and evaluation 
literature. The survey is intended to assist school professional staff 
in ascertaining how well positioned schools are to undertake 
school reform efforts. It is also intended for administration and 
analysis over the course of school improvement undertakings. The 
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instrument has been pilot-tested, revised, and field-tested to estab- 
lish its validity and reliability. 11 

AEL evaluators administered the AEL MSCI. It took up to 25 
minutes for the professional school staff to complete the 64-item 
survey. For 31 items, professional staff were asked to rate the ex- 
tent to which each item was true for their school using a 6-point 
Likert scale ranging from 1 (Not at all true) to 6 ( Completely true). 
For the remaining items, professional staff were asked to rate how 
often each item was true for their school using a similar 6-point 
Likert scale ranging from 1 ( Never true) to 6 (. Always true). The 
MSCI subscale scores were created by calculating the mean of the 
items relating to each subscale; thus, subscale scores range from 
1.0 to 6.0. 

MSCI subscales are described below: 

1. Collective Professional Capacity subscale measures the 
extent to which a faculty believes in its shared capability to 
positively influence student learning. 

2. Peer-Reviewed Practice assesses the frequency with 
which teachers and supervisors observe a staff member’s 
classes to provide meaningful feedback and improve teach- 
ing. 

3. Program Coherence evaluates the extent to which the 
school’s programs for student and staff learning are coordi- 
nated, focused on clear learning goals, and sustained over 
time. 

4. Technical Resources measures the availability to faculty 
of planning time, working equipment, technology, instruc- 
tional materials, facilities, and professional resource mate- 
rials, such as journals. 



11 Howley, C., and Riffle, J., 2002, Pilot test of AEL’ s school capacity assessment, 
Charleston, WV : AEL. 

Riffle, M. J., Howley, C. W., & Ermolov, L. D. (2004, April). Measure of school 
capacity for improvement (MSCI): Early field test findings. Paper presented at 
the annual meeting of the American Educational Research Association, San 
Diego, CA. 
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5. Anti-Discriminatory Teaching subscale assesses the de- 
gree to which teachers instruct students in how to confront 
or resist prejudice and discrimination. 

6. Responsive Pedagogy subscale is composed of items that 
concern faculty responsiveness to students’ communities, 
the creation of equitable classroom environments, and plu- 
ralistic language and text use. 

7. Differentiated Instruction evaluates the extent to which a 
faculty modifies its instructional strategies and grouping ar- 
rangements to meet the learning needs of students. 

8. Student Performance Expectations measures how aca- 
demically capable a staff believes its students are and how 
well it expects its students to perform. 

Table 9 shows the descriptive statistics on the eight subscales of 
the MSCI survey for 313 Petersburg professional staff who com- 
pleted the MSCI survey instrument in 2003, and for the 282 who 
completed the instrument in 2004. Three of the schools (B, D, and 
G) showed increasing mean scores on three to four of the eight 
MSCI subscales, albeit each school did not increase on the same 
group of subscales. All these schools improved on the anti- 
discriminatory teaching and differentiated instruction scales. 
School B also had an improved mean score on the responsive 
pedagogy scale. School D showed improved mean scores on col- 
lective teacher efficacy and student performance expectations 
scales. And School G showed improved mean scores also on col- 
lective teacher efficacy and student performance. 



Table 9 Descriptive Statistics for AEL MSCI Survey of Capacity for Improvement 





Mean 


Median 


Std. Dev. 


2003 


2004 


2003 


2004 


2003 


2004 


School A 


n= 26 


//= 25 










Collective teacher efficacy 


4.7 


4.7 


4.7 


4.9 


0.8 


0.7 


Peer reviewed practice 


4.4 


4.1 


4.5 


4.0 


0.8 


0.9 


Program coherence 


4.6 


4.3 


4.6 


4.3 


0.8 


0.8 


Technical resources 


4.3 


4.2 


4.3 


4.3 


1.0 


0.9 


Anti-discriminatory teaching 


5.3 


5.0 


5.3 


5.0 


0.6 


0.8 


Responsive pedagogy 


4.9 


4.8 


4.9 


4.8 


0.6 


0.8 
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Table 9 Descriptive Statistics for AEL MSCI Survey of Capacity for Improvement (continued) 





Mean 


Median 


Std. Dev. 


2003 


2004 


2003 


2004 


2003 


2004 


Differentiated instruction 


4.8 


4.9 


4.8 


5.0 


0.8 


0.8 


Student performance expectations 


4.8 


4.8 


4.7 


4.8 


0.8 


0.8 


School B 


«=16 


//=16 










Collective teacher efficacy 


5.0 


4.9 


5.0 


5.0 


0.7 


0.5 


Peer reviewed practice 


4.4 


4.3 


4.5 


4.5 


1.2 


0.9 


Program coherence 


4.6 


4.5 


4.8 


4.6 


1.1 


0.7 


Technical resources 


4.3 


4.3 


4.5 


4.2 


1.0 


0.5 


Anti-discriminatory teaching 


5.1 


5.2 


5.6 


5.2 


1.1 


0.6 


Responsive pedagogy 


4.8 


5.1 


5.2 


5.1 


1.2 


0.4 


Differentiated instruction 


4.8 


5.1 


5.3 


5.0 


1.0 


0.8 


Student performance expectations 


5.1 


5.1 


5.2 


5.4 


0.8 


0.8 


School C 


r- 

fN 

II 

si 


«=17 










Collective teacher efficacy 


4.7 


4.9 


4.7 


5.0 


0.7 


0.6 


Peer reviewed practice 


4.5 


4.4 


4.5 


4.6 


0.8 


0.9 


Program coherence 


4.7 


4.5 


4.7 


4.3 


0.5 


0.8 


Technical resources 


4.0 


4.0 


4.1 


4.0 


0.8 


0.8 


Anti-discriminatory teaching 


5.1 


5.2 


5.3 


5.5 


0.8 


0.6 


Responsive pedagogy 


4.9 


4.8 


4.8 


5.0 


0.6 


0.7 


Differentiated instruction 


4.8 


5.0 


4.9 


4.9 


0.8 


0.6 


Student performance expectations 


4.8 


5.1 


4.9 


5.1 


0.9 


0.6 


School D 


«=30 


//= 26 










Collective teacher efficacy 


5.0 


4.7 


5.1 


4.9 


0.6 


0.9 


Peer reviewed practice 


4.6 


4.1 


4.8 


4.1 


0.9 


1.3 


Program coherence 


4.8 


4.3 


5.1 


4.5 


0.9 


0.8 


Technical resources 


3.7 


3.9 


3.8 


4.1 


0.9 


1.1 


Anti-discriminatory teaching 


5.4 


5.0 


5.6 


5.4 


0.7 


1.2 


Responsive pedagogy 


5.1 


4.9 


5.3 


5.2 


0.7 


0.9 


Differentiated instruction 


5.1 


4.7 


5.1 


4.9 


0.8 


1.0 


Student performance expectations 


5.1 


4.8 


5.3 


5.1 


0.7 


1.0 


School E 


n= 25 


;/=19 










Collective teacher efficacy 


4.5 


4.3 


4.3 


4.5 


0.9 


1.0 
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Table 9 Descriptive Statistics for AEL MSCI Survey of Capacity for Improvement (continued) 





Mean 


Median 


Std. Dev. 


2003 


2004 


2003 


2004 


2003 


2004 


Peer reviewed practice 


4.2 


4.2 


4.5 


4.5 


1.0 


1.1 


Program coherence 


4.4 


3.8 


4.5 


3.9 


0.8 


0.9 


Technical resources 


3.7 


3.5 


3.9 


3.6 


1.1 


0.7 


Anti-discriminatory teaching 


4.8 


4.5 


4.7 


4.5 


0.9 


0.9 


Responsive pedagogy 


4.6 


4.3 


4.7 


4.1 


0.9 


0.8 


Differentiated instruction 


4.5 


4.5 


4.5 


4.6 


1.2 


1.1 


Student performance expectations 


4.6 


4.3 


4.5 


4.5 


1.1 


1.1 


School F 


n= 41 


//= 29 










Collective teacher efficacy 


4.5 


4.4 


4.5 


4.4 


0.8 


0.9 


Peer reviewed practice 


4.1 


4.3 


4.3 


4.5 


1.0 


1.1 


Program coherence 


4.3 


4.0 


4.3 


4.1 


0.7 


0.7 


Technical resources 


3.8 


3.8 


3.9 


3.7 


0.9 


1.1 


Anti-discriminatory teaching 


5.0 


5.1 


5.3 


5.3 


0.9 


0.9 


Responsive pedagogy 


4.8 


4.8 


5.0 


4.9 


0.8 


0.9 


Differentiated instruction 


4.5 


4.5 


4.5 


4.6 


0.9 


1.0 


Student performance expectations 


4.8 


4.4 


4.7 


4.5 


0.8 


0.9 


School G 


m =21 


n=24 










Collective teacher efficacy 


4.8 


4.9 


4.9 


4.9 


0.8 


0.5 


Peer reviewed practice 


4.7 


4.7 


4.5 


4.7 


0.8 


0.7 


Program coherence 


4.9 


4.5 


5.3 


4.4 


0.7 


0.6 


Technical resources 


4.6 


4.2 


4.5 


4.3 


0.9 


0.9 


Anti-discriminatory teaching 


5.2 


5.3 


5.4 


5.3 


0.7 


0.6 


Responsive pedagogy 


5.0 


5.0 


4.7 


4.9 


0.7 


0.6 


Differentiated instruction 


4.8 


5.1 


4.9 


5.1 


0.8 


0.6 


Student performance expectations 


4.9 


5.1 


5.1 


5.4 


0.9 


0.6 


School H 


M =0 


n=34 










Collective teacher efficacy 




4.5 




4.6 




1.1 


Peer reviewed practice 




4.5 




4.7 




1.0 


Program coherence 




4.4 




4.3 




1.1 


Technical resources 




3.7 




3.7 




1.1 


Anti-discriminatory teaching 




5.0 




5.1 




1.0 


Responsive pedagogy 




4.6 




4.8 




0.9 
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Table 9 Descriptive Statistics for AEL MSCI Survey of Capacity for Improvement (continued) 





Mean 


Median 


Std. Dev. 


2003 


2004 


2003 


2004 


2003 


2004 


Differentiated instruction 




4.6 




4.9 




1.1 


Student performance expectations 




4.4 




4.5 




1.1 


School I 


n= 29 


Tf 

Tf 

II 

5* 










Collective teacher efficacy 


4.3 


4.1 


4.5 


4.2 


1.0 


0.9 


Peer reviewed practice 


4.2 


4.1 


4.3 


4.3 


1.1 


1.1 


Program coherence 


4.2 


4.0 


4.5 


4.1 


1.1 


0.9 


Technical resources 


3.8 


3.8 


3.9 


4.1 


1.0 


1.1 


Anti-discriminatory teaching 


4.8 


4.8 


5.1 


4.8 


1.0 


0.9 


Responsive pedagogy 


4.7 


4.5 


4.7 


4.6 


1.0 


0.9 


Differentiated instruction 


4.3 


4.3 


4.5 


4.3 


1.1 


0.9 


Student performance expectations 


4.3 


4.2 


4.3 


4.3 


1.2 


1.0 


School J 


n= 90 


m=41 










Collective teacher efficacy 


4.2 


3.9 


4.3 


4.0 


0.9 


1.0 


Peer reviewed practice 


4.2 


3.9 


4.1 


4.0 


0.9 


1.0 


Program coherence 


4.1 


3.4 


4.2 


3.8 


0.8 


0.9 


Technical resources 


3.7 


3.0 


3.7 


3.3 


0.9 


1.0 


Anti-discriminatory teaching 


4.8 


4.4 


4.8 


4.9 


0.8 


1.1 


Responsive pedagogy 


4.4 


4.1 


4.5 


4.5 


0.8 


1.1 


Differentiated instruction 


4.2 


3.7 


4.3 


3.8 


1.0 


1.1 


Student performance expectations 


4.3 


4.1 


4.5 


3.9 


1.0 


1.1 


School K 


n= 0 


n=l 










Collective teacher efficacy 




4.6 




4.8 




0.5 


Peer reviewed practice 




4.1 




4.0 




0.7 


Program coherence 




3.2 




3.1 




0.7 


Technical resources 




3.1 




3.0 




0.4 


Anti-discriminatory teaching 




5.1 




5.0 




0.4 


Responsive pedagogy 




4.8 




4.8 




0.5 


Differentiated instruction 




4.8 




4.9 




0.6 


Student performance expectations 




4.7 




4.9 




0.7 


School L 


n=l 


n= 0 










Collective teacher efficacy 


3.6 


0.0 


3.6 


0.0 


0.9 


0.0 


Peer reviewed practice 


3.7 


0.0 


3.2 


0.0 


1.0 


0.0 
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Table 9 Descriptive Statistics for AEL MSCI Survey of Capacity for Improvement (continued) 





Mean 


Median 


Std. Dev. 


2003 


2004 


2003 


2004 


2003 


2004 


Program coherence 


3.6 


0.0 


3.9 


0.0 


0.5 


0.0 


Technical resources 


2.8 


0.0 


2.8 


0.0 


0.7 


0.0 


Anti-discriminatory teaching 


4.8 


0.0 


4.7 


0.0 


0.8 


0.0 


Responsive pedagogy 


4.6 


0.0 


5.3 


0.0 


1.2 


0.0 


Differentiated instruction 


3.9 


0.0 


4.1 


0.0 


1.0 


0.0 


Student performance expectations 


3.8 


0.0 


3.8 


0.0 


1.3 


0.0 



A total of 94 professional staff completed the MSCI survey in both 
2003 and 2004. Summary statistics presented in Table 10 were 
compiled for the eight scales on the MSCI for the /-tests calculated. 
Table 1 1 shows that findings are significant for matched paired t- 
tests on three of the eight scales: program coherence, anti- 
discriminatory teaching, and responsive pedagogy. Because mean 
survey scores were higher in 2003 than in 2004, the statistical rela- 
tionship shown for these three scales indicates a significant de- 
crease in school capacity for improvement in these three areas 
when the school division as a whole is examined. 



Table 10 Descriptive Statistics for Matched Pairs for AEL MSCI Survey of Capacity for Improve- 
ment 





Mean 


Std. Dev. 


2003 


2004 


2003 


2004 


Collective teacher efficacy 


4.5 


4.5 


0.8 


0.9 


Peer reviewed practice 


4.2 


4.1 


1.0 


1.0 


Program coherence 


4.4 


4.1 


0.8 


1.0 


Technical resources 


3.9 


3.7 


1.0 


1.1 


Anti-discriminatory teaching 


5.1 


4.9 


0.8 


0.9 


Responsive pedagogy 


4.8 


4.6 


0.8 


0.8 


Differentiated instruction 


4.5 


4.5 


1.0 


0.9 


Student performance expectations 


4.7 


4.6 


0.9 


0.9 
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Table 11 Results of Paired t-Test and Cohen’s d Estimates of Effect Size for AEL Measure of School 
Capacity for Improvement 





Paired Differences 


Test Statistic 


Two-tailed 

Significance 


Effect 

Size 

id) 


Mean 


Std. Dev. 


Collective teacher efficacy 


0.0 


0.8 


0.5 


0.59 




Peer reviewed practice 


0.1 


0.9 


1.0 


0.30 




Program coherence 


0.4 


0.7 


5.0 


<.001 


0.51 


Technical resources 


0.2 


0.9 


1.8 


0.07 


0.19 


Anti-discriminatory teach- 
ing 


0.2 


0.8 


2.3 


0.02 


0.24 


Responsive pedagogy 


0.2 


0.7 


2.8 


<.001 


0.25 


Differentiated instruction 


0.0 


0.8 


0.0 


0.96 




Student performance ex- 
pectations 


0.0 


0.8 


1.0 


0.34 





Note: The mean is based on the 2003-2004 value, so a positive mean indicates a decrease in subscale 

score. Significant values indicate that the mean is significantly different from zero and, in this 
case, indicate a significant decline in subscale score. 
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Standards of Learning 

The Standards of Learning (SOLs) for Virginia Public Schools de- 
fine the Commonwealth’s expectations for student learning and 
achievement in grades K through 12. SOL tests cover English, 
mathematics, science, and history/social science. The Virginia 
Public Schools Web site indicates that materials examined repre- 
sent a broad consensus of what parents, classroom teachers, school 
administrators, academics, and business and community leaders 
believe schools should teach and students should master. In the 
four core areas (English, mathematics, science, and history/social 
science), the Commonwealth provides teachers with curriculum 
frameworks that detail the specific knowledge and skills students 
must demonstrate to meet standards of learning for these sub- 
jects. 12 

For each school, SOL and end-of-course test results are reported in 
terms of the percentages of students passing (i.e., scoring at profi- 
cient or advanced) each subject area test. These pass rates are 
available from the VDOE Web site. 13 

To evaluate the impact of the Model IV Intervention program, the 
10 Petersburg City Schools are paired with similar schools in the 
City of Richmond School Division. The Richmond City schools 
are the control group for the Petersburg schools (i.e., because there 
is no intervention program in the Richmond City schools). CNAC 
developed the school-matching criteria used to select schools in 
Richmond City School Division for the comparison group. 

SOL pass rates at the school level were downloaded from the Vir- 
ginia Department of Education Web site. Again, the SOL results 
show the percentages of students passing the SOL — by grade and 
for each subject tested. Data used for this evaluation cover the five 
years prior to the school improvement intervention (1998, 1999, 
2000, 2001, and 2002) and the first year of the three-year interven- 
tion period (2003) for which there are data. The purpose is to 
compare SOL results for the baseline years (1998 to 2002, before 
the intervention program) with the intervention year (2003) for 
those areas of student achievement targeted by the Petersburg City 



12 www.pen.kl2.va.us/VDOE/Superintendent/Sols/home.shtml 

1 3 http ://www.pen.k 1 2,va.us/VDOE/Assessment/2002 SOLpassrates.html 
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Schools’ school improvement plan. The SOL pass rates are shown 
for: (1) third-grade English, mathematics, history, and science; (2) 
fifth- and eighth-grade English, writing, mathematics, history, and 
science; and (3) high school English, writing, Algebra I & II, ge- 
ometry, U.S. History I & II, World History I & II, world geogra- 
phy, earth science, biology, and chemistry. Because school-level 
pass rates are used, inferential statistics and effect sizes could not 
be used. The intention is to compare trends in student perfonnance 
across the years between each Petersburg school and its matched 
control school. 

Table 12 shows changes in SOL pass rates for third-grade English, 
math, history, and science for Petersburg schools for the five years 
prior to the intervention (1998 to 2002) and Model IV Intervention 
year (2003). Comparable SOL data for the comparison schools are 
also indicated. 



Table 12 Changes in Percentages of Students Passing SOLs for Third-Grade Subjects 





Baseline 


Intervention 


1998 


1999 


2000 


2001 


2002 


1998-2002 

change 


1998- 
2002 % 
change 


2003 


2002-2003 

change 


2002-2003 
% change 


Third-Grade English 






















School A 


30.0 


25.0 


27.1 


28.0 


38.6 


8.6 


28.8% 


40.0 


1.4 


3.5% 


School A Control 


37.0 


30.4 


24.5 


53.7 


43.6 


6.7 


18.1% 


40.7 


-3.0 


-6.8% 


School B 






33.3 


19.6 


38.9 






59.3 


20.4 


52.4% 


School B Control 


38.8 


48.9 


25.9 


47.1 


39.3 


0.5 


1.3% 


33.3 


-6.0 


-15.2% 


School C 


20.3 


26.6 


35.3 


24.2 


45.7 


25.4 


125.0% 


36.1 


-9.5 


-20.9% 


School C Control 


18.6 


11.4 


16.1 


23.5 


38.8 


20.2 


108.4% 


50.0 


11.2 


28.9% 


School D 


43.8 


34.2 


26.4 


27.3 


45.7 


1.9 


4.4% 


41.0 


-4.7 


-10.3% 


School D Control 


19.0 


31.0 


38.3 


31.4 


78.2 


59.1 


310.5% 


70.4 


-7.8 


-10.0% 


School E 


28.7 


45.9 


55.2 


28.8 


44.1 


15.3 


53.4% 


51.1 


7.0 


15.9% 


School E Control 


37.8 


17.2 


13.8 


10.7 


45.1 


7.3 


19.4% 


53.2 


8.1 


17.9% 


School F 


30.1 


35.9 


33.1 


39.6 


53.7 


23.6 


78.5% 


56.7 


3.0 


5.6% 


School F Control 


33.3 


60.7 


42.1 


41.4 


67.2 


33.9 


101.7% 


63.8 


-3.5 


-5.2% 


School G 


20.0 


20.6 


12.3 


16.7 


39.7 


19.7 


98.3% 


49.0 


9.3 


23.5% 


School G Control 


33.3 


33.9 


25.0 


21.2 


28.9 


-4.4 


-13.3% 


36.5 


7.6 


26.5% 


Third-Grade Math 






















School A 


49.0 


28.0 


32.7 


34.0 


44.4 


-4.5 


-9.3% 


57.5 


13.1 


29.4% 


School A Control 


34.8 


37.5 


47.2 


51.9 


69.5 


34.7 


99.8% 


91.9 


22.4 


32.3% 


School B 






35.2 


30.4 


22.6 






74.1 


51.4 


227.2% 


School B Control 


50.0 


55.3 


38.2 


50.0 


34.4 


-15.6 


-31.1% 


47.6 


13.2 


38.3% 


School C 


44.9 


27.8 


47.1 


36.4 


39.1 


-5.8 


-12.9% 


57.1 


18.0 


46.0% 


School C Control 


32.6 


13.3 


12.9 


31.4 


47.3 


14.7 


45.2% 


70.8 


23.6 


49.8% 


School D 


48.0 


30.2 


33.8 


38.6 


41.3 


-6.8 


-14.1% 


53.5 


12.2 


29.7% 


School D Control 


12.2 


27.6 


40.3 


50.0 


58.9 


46.7 


383.2% 


78.6 


19.6 


33.3% 


School E 


50.0 


45.9 


59.3 


36.1 


47.5 


-2.5 


-5.1% 


68.1 


20.6 


43.5% 
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Table 12 Changes in Percentages of Students Passing SOLs for Third-Grade Subjects (continued) 





Baseline 


Intervention 


1998 


1999 


2000 


2001 


2002 


1998-2002 

change 


1998- 
2002 % 
change 


2003 


2002-2003 

change 


2002-2003 
% change 


Third-Grade Math 






















School E Control 


34.8 


17.2 


20.6 


27.1 


61.2 


26.4 


76.0% 


64.0 


2.8 


4.5% 


School F 


46.4 


34.0 


33.8 


44.4 


66.7 


20.2 


43.6% 


62.0 


-4.7 


-7.0% 


School F Control 


46.7 


50.8 


57.9 


54.8 


75.9 


29.2 


62.6% 


80.8 


5.0 


6.5% 


School G 


38.5 


25.0 


20.3 


31.1 


42.9 


4.4 


11.4% 


61.2 


18.4 


42.9% 


School G Control 


37.8 


37.5 


41.7 


37.0 


37.8 


0.0 


0.0% 


60.7 


22.9 


60.6% 


Third-Grade History 






















School A 


20.4 


42.9 


20.4 


26.0 


44.7 


24.3 


118.9% 


60.0 


15.3 


34.3% 


School A Control 


8.7 


20.0 


17.0 


43.6 


52.7 


44.0 


506.4% 


87.9 


35.2 


66.8% 


School B 






16.7 


23.9 


46.3 






75.9 


29.6 


64.0% 


School B Control 


52.0 


47.8 


43.6 


51.0 


40.0 


-12.0 


-23.1% 


35.3 


-4.7 


-11.8% 


School C 


14.5 


24.1 


52.9 


42.4 


56.5 


42.0 


290.0% 


33.3 


-23.2 


-41.0% 


School C Control 


16.3 


2.3 


3.2 


51.0 


47.9 


31.6 


194.3% 


60.4 


12.5 


26.1% 


School D 


31.5 


21.6 


18.1 


25.8 


39.5 


8.0 


25.3% 


62.4 


22.8 


57.8% 


School D Control 


9.4 


24.4 


37.1 


41.1 


71.4 


62.0 


658.9% 


74.5 


3.1 


4.4% 


School E 


38.6 


41.7 


42.4 


25.4 


27.6 


-11.1 


-28.6% 


59.6 


32.0 


116.0% 


School E Control 


26.7 


14.1 


27.3 


32.1 


56.3 


29.6 


110.9% 


85.1 


28.9 


51.3% 


School F 


27.5 


33.3 


26.9 


45.1 


54.3 


26.8 


97.5% 


64.2 


9.9 


18.3% 


School F Control 


26.7 


35.0 


67.8 


78.0 


76.7 


50.0 


187.5% 


83.3 


6.7 


8.7% 


School G 


12.3 


19.4 


10.2 


8.2 


38.6 


26.3 


213.6% 


55.1 


16.5 


42.8% 


School G Control 


25.0 


38.6 


33.3 


43.6 


24.4 


-0.6 


-2.2% 


36.7 


12.2 


50.0% 


Third-Grade Science 






















School A 


30.0 


20.4 


26.5 


34.0 


36.2 


6.2 


20.6% 


66.7 


30.5 


84.3% 


School A Control 


21.7 


25.0 


35.8 


47.3 


44.4 


22.7 


104.4% 


96.6 


52.1 


117.2% 


School B 






33.3 


30.4 


35.2 






70.4 


35.2 


100.0% 


School B Control 


55.3 


36.2 


41.5 


51.9 


43.4 


-11.9 


-21.6% 


39.0 


-4.4 


-10.1% 


School C 


34.8 


25.3 


55.9 


27.3 


47.8 


13.0 


37.5% 


57.1 


9.3 


19.5% 


School C Control 


26.2 


4.8 


24.2 


49.0 


60.4 


34.2 


130.7% 


66.7 


6.3 


10.3% 


School D 


40.4 


26.4 


28.8 


31.5 


38.1 


-2.3 


-5.7% 


60.7 


22.6 


59.4% 


School D Control 


21.4 


35.6 


54.1 


46.4 


68.3 


46.9 


218.9% 


82.1 


13.8 


20.2% 


School E 


39.4 


58.3 


57.6 


38.1 


41.4 


2.0 


5.1% 


60.8 


19.4 


46.9% 


School E Control 


34.9 


12.5 


27.7 


30.2 


49.0 


14.1 


40.4% 


66.7 


17.7 


36.1% 


School F 


42.0 


37.9 


29.2 


47.2 


61.7 


19.7 


47.1% 


64.2 


2.5 


4.0% 


School F Control 


31.1 


55.0 


53.4 


61.0 


82.8 


51.6 


166.0% 


76.7 


-6.0 


-7.3% 


School G 


23.1 


17.9 


10.2 


24.6 


49.1 


26.0 


112.9% 


58.8 


9.7 


19.7% 


School G Control 


37.8 


32.1 


37.5 


24.5 


37.8 


0.0 


0.0% 


47.3 


9.5 


25.1% 
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At the third-grade level, during the first year of Model IV Interven- 
tion, the following trends were found: 

• Students in 5 of the 7 Petersburg elementary schools 
(Schools A, B, E, F, and G) show gains in the percentages 
of students passing English SOLs. 

• Students in 6 of the 7 Petersburg elementary schools 
(Schools A, B, C, D, E, and G) show gains in the percent- 
ages of students passing math SOLs. 

• Students in 6 of the 7 Petersburg elementary schools 
(Schools A, B, D, E, F, and G) show gains in the percent- 
ages of students passing history SOLs. 

• All elementary schools show gains in the percentages of 
students passing science SOLs. 

Comparisons in SOL test results between third graders in 
Richmond control schools and third graders in Petersburg schools 
during the first intervention year produced the following results: 

• Petersburg elementary schools A, B, and F (compared to 
their matched comparison Richmond schools) show a 
greater increase in the percentages of students passing 
SOLs in English. 

• Petersburg elementary schools B and E show a greater in- 
crease in the percentages of students passing SOLs in math. 

• Petersburg elementary schools B, D, E, and F show a 
greater increase in the percentages of students passing 
SOLs in history. 

Table 13 shows changes in SOL pass rates for fifth-grade English, 
math, history, and science for Petersburg schools for the five years 
prior to the intervention (1998 to 2002) and the Model IV Interven- 
tion year (2003). Comparable SOL data for the comparison 
schools are also indicated. 
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Table 13 Changes in Percentages of Students Passing SOLs for Fifth-Grade Subjects 





Baseline 


Intervention 




1998 


1999 


2000 


2001 


2002 


1998-2002 

change 


1998- 
2002 % 
change 


2003 


2002-2003 

change 


2002-2003 % 
change 


Fifth-Grade Writing 






















School A 


31.4 


36.7 


60.5 


59.1 


70.6 


39.2 


125.0% 


58.3 


-12.3 


-17.4% 


School A Control 


33.3 


41.2 


44.2 


57.4 


59.6 


26.2 


78.7% 


87.2 


27.7 


46.4% 


School B 






76.7 


63.3 


70.0 






73.6 


3.6 


5.1% 


School B Control 


61.9 


83.3 


47.2 


82.5 


69.8 


7.9 


12.7% 


89.2 


19.4 


27.8% 


School C 


26.2 


55.2 


50.0 


68.8 


56.9 


30.7 


117.1% 


41.8 


-15.0 


-26.5% 


School C Control 


32.7 


56.3 


48.6 


39.4 


35.4 


2.8 


8.5% 


60.5 


25.0 


70.7% 


School D 


39.1 


63.7 


53.8 


54.3 


39.1 


0.0 


0.0% 


50.0 


10.9 


28.0% 


School D Control 


47.7 


56.1 


67.8 


95.3 


60.9 


13.2 


27.7% 


84.1 


23.2 


38.1% 


School E 


48.2 


60.4 


65.2 


80.0 


55.1 


6.9 


14.3% 


45.3 


-9.8 


-17.8% 


School E Control 


14.6 


50.0 


42.9 


30.4 


32.4 


17.8 


122.4% 


45.8 


13.4 


41.3% 


School F 


38.5 


60.6 


65.8 


66.9 


65.2 


26.7 


69.3% 


73.3 


8.0 


12.3% 


School F Control 


41.5 


69.2 


86.9 


89.7 


79.4 


37.9 


91.3% 


82.8 


3.3 


4.2% 


School G 


31.3 


67.6 


76.9 


58.5 


51.8 


20.5 


65.7% 


52.0 


0.2 


0.4% 


School G Control 


41.7 


73.8 


54.8 


59.2 


40.5 


-1.2 


-2.9% 


68.9 


28.4 


70.2% 


Fifth-Grade English 






















School A 


49.0 


25.0 


37.8 


17.1 


45.9 


-3.1 


-6.3% 


56.8 


10.8 


23.5% 


School A Control 


26.0 


28.0 


21.4 


35.2 


41.3 


15.3 


58.9% 


72.9 


31.6 


76.5% 


School B 






50.0 


38.3 


52.6 






51.9 


-0.7 


-1.3% 


School B Control 


38.1 


40.0 


26.4 


61.0 


48.7 


10.6 


27.9% 


76.3 


27.6 


56.6% 


School C 


30.0 


41.7 


30.0 


34.4 


50.9 


20.9 


69.6% 


35.7 


-15.2 


-29.8% 


School C Control 


26.0 


20.4 


22.5 


34.4 


25.9 


-0.1 


-0.5% 


84.4 


58.6 


226.5% 


School D 


53.3 


49.5 


32.5 


25.6 


34.8 


-18.5 


-34.7% 


38.7 


3.8 


11.0% 


School D Control 


36.0 


37.5 


36.7 


56.3 


46.5 


10.6 


29.4% 


71.7 


25.2 


54.2% 


School E 


51.7 


58.7 


31.8 


42.0 


27.7 


-24.1 


-46.5% 


43.4 


15.7 


56.9% 


School E Control 


13.3 


18.2 


17.2 


20.4 


11.4 


-2.0 


-14.8% 


42.0 


30.6 


269.6% 


School F 


44.2 


43.5 


21.6 


32.8 


53.4 


9.3 


21.0% 


78.4 


25.0 


46.7% 


School F Control 


48.1 


29.4 


71.7 


41.4 


78.0 


29.9 


62.2% 


60.7 


-17.3 


-22.1% 


School G 


32.3 


57.1 


29.7 


33.3 


32.2 


-0.1 


-0.3% 


56.9 


24.7 


76.6% 


School G Control 


25.4 


25.6 


36.6 


33.3 


34.1 


8.7 


34.1% 


59.6 


25.5 


74.8% 


Fifth-Grade Math 






















School A 


23.5 


12.5 


29.7 


9.5 


21.1 


-2.5 


-10.5% 


40.5 


19.5 


92.6% 


School A Control 


14.0 


2.0 


23.8 


14.8 


60.9 


46.9 


334.8% 


79.2 


18.4 


30.2% 


School B 






45.2 


25.5 


17.9 






32.1 


14.2 


79.6% 


School B Control 


25.6 


40.0 


32.1 


41.5 


59.0 


33.4 


130.5% 


75.0 


16.0 


27.2% 


School C 


10.0 


5.0 


17.5 


25.0 


24.6 


14.6 


145.6% 


21.4 


-3.1 


-12.8% 


School C Control 


26.0 


4.1 


20.5 


28.1 


28.0 


2.0 


7.7% 


74.0 


46.0 


164.3% 


School D 


18.7 


11.1 


37.0 


17.8 


13.4 


-5.2 


-28.1% 


40.3 


26.8 


199.8% 


School D Control 


18.2 


14.1 


25.0 


52.9 


63.6 


45.5 


250.0% 


71.4 


7.8 


12.2% 
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Table 13 Changes in Percentages of Students Passing SOLs for Fifth-Grade Subjects (continued) 





Baseline 


Intervention 




1998 


1999 


2000 


2001 


2002 


1998-2002 

change 


1998- 
2002 % 
change 


2003 


2002-2003 

change 


2002-2003 % 
change 


School E 


25.9 


28.3 


22.7 


29.4 


32.6 


6.7 


26.1% 


35.8 


3.2 


9.9% 


School E Control 


10.9 


4.7 


16.9 


10.2 


16.2 


5.3 


49.2% 


16.4 


0.1 


0.9% 


School F 


13.1 


8.3 


19.0 


23.9 


37.6 


24.5 


186.8% 


50.9 


13.3 


35.5% 


School F Control 


9.6 


19.6 


69.4 


41.4 


86.2 


76.6 


796.6% 


40.0 


-46.2 


-53.6% 


School G 


6.2 


18.4 


10.9 


12.3 


11.5 


5.3 


86.5% 


73.1 


61.6 


536.8% 


School G Control 


11.9 


20.9 


43.9 


31.3 


31.1 


19.2 


162.2% 


72.9 


41.8 


134.4% 


Fifth-Grade History 






















School A 


11.8 


11.6 


0.0 


6.8 


27.8 


16.0 


136.1% 








School A Control 


8.0 


4.0 


6.1 


5.2 


56.6 


48.6 


607.5% 


84.6 


28.0 


49.5% 


School B 






5.5 


15.8 


41.4 












School B Control 


9.3 


22.9 


16.2 


19.2 


29.2 


19.9 


214.2% 


69.0 


39.8 


136.2% 


School C 


5.0 


7.9 


28.6 


57.1 


46.2 


41.2 


823.1% 








School C Control 


2.0 


2.0 


5.1 


27.6 


48.8 


46.8 


2341.9% 


72.5 


23.7 


48.5% 


School D 


4.5 


13.0 


16.0 


6.9 


23.5 


19.0 


421.9% 








School D Control 


7.9 


4.7 


15.0 


26.7 


67.2 


59.4 


754.9% 


89.6 


22.3 


33.2% 


School E 


32.8 


25.2 


8.6 


17.4 


35.5 


2.7 


8.3% 








School E Control 


4.4 


9.1 


0.9 


0.0 


13.5 


9.0 


202.9% 


38.1 


24.6 


183.0% 


School F 


5.6 


8.6 


4.2 


31.2 


67.9 


62.2 


1102.0% 








School F Control 


17.3 


20.8 


21.7 


37.3 


59.3 


42.0 


242.7% 


90.5 


31.2 


52.5% 


School G 


3.1 


19.9 


7.9 


6.5 


6.3 


3.3 


106.4% 








School G Control 


1.7 


27.9 


15.0 


18.6 


30.8 


29.1 


1715.4% 


21.6 


-9.1 


-29.7% 


Fifth-Grade Science 






















School A 


27.5 


10.4 


8.1 


31.0 


15.4 


-12.1 


-44.0% 


30.6 


15.2 


98.6% 


School A Control 


20.0 


12.0 


9.5 


28.3 


37.0 


17.0 


84.8% 


87.5 


50.5 


136.8% 


School B 






45.2 


25.5 


21.4 






34.6 


13.2 


61.5% 


School B Control 


20.9 


57.1 


25.5 


58.5 


50.0 


29.1 


138.9% 


57.9 


7.9 


15.8% 


School C 


20.0 


18.3 


15.0 


37.5 


24.6 


4.6 


22.8% 


21.8 


-2.7 


-11.2% 


School C Control 


18.4 


14.3 


5.0 


31.3 


38.0 


19.6 


106.9% 


42.3 


4.3 


11.3% 


School D 


25.9 


25.8 


27.8 


44.4 


22.1 


-3.8 


-14.8% 


47.9 


25.9 


117.4% 


School D Control 


30.7 


18.8 


18.3 


72.5 


54.3 


23.7 


77.1% 


76.6 


22.2 


40.9% 


School E 


40.4 


42.2 


21.5 


47.1 


22.4 


-17.9 


-44.4% 


28.3 


5.9 


26.1% 


School E Control 


9.1 


13.6 


3.5 


21.3 


10.5 


1.4 


15.8% 


38.0 


27.5 


261.0% 


School F 


28.2 


27.5 


13.8 


50.0 


47.4 


19.1 


67.8% 


51.5 


4.1 


8.7% 


School F Control 


28.8 


30.2 


59.7 


51.4 


72.6 


43.7 


151.6% 


59.4 


-13.2 


-18.2% 


School G 


20.3 


36.8 


14.1 


31.6 


30.0 


9.7 


47.7% 


73.6 


43.6 


145.3% 


School G Control 


18.6 


21.4 


29.3 


34.8 


24.4 


5.8 


31.1% 


52.7 


28.3 


115.7% 
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Table 13 shows generally positive gains in the percentages of fifth- 
grade students passing SOLs in each subject during the Model IV 
Intervention year (2003). However, Schools A, C, and E show de- 
creases in pass rates for the writing SOL test, and Schools B and C 
show decreases in pass rates for the English SOL test. In addition, 
School C shows decreases in the percentages of students passing 
SOLs in math and science. 

Lor the fifth-grade SOLs, most Richmond control schools show 
greater increases in percentages of pass rates (compared to the Pe- 
tersburg schools) for the SOLs in writing and English; this pattern 
begins to reverse in the math and science SOLs, where the Peters- 
burg schools seem to be making greater gains than the Richmond 
schools. The SOL data for history SOLs are incomplete. 

Table 14 shows changes in SOL pass rates for Petersburg schools 
eighth-grade writing, English, math, history, and science Peters- 
burg schools’ for the five years prior to the intervention (1998 to 
2002) and the Model IV Intervention period (2003). Comparable 
SOL data for the comparison schools are also indicated. 



Table 14 Changes in Percentages of Students Passing SOLs for Eighth-Grade Subjects 





Baseline 


Intervention 


1998 


1999 


2000 


2001 


2002 


1998-2002 

change 


1998-2002 % 
change 


2003 


2002-2003 

change 


2002-2003 
% change 


Eighth-Grade Writing 






















School H 


38.5 


40.8 


51.8 


51.0 


36.0 


-2.5 


-6.5% 


32.8 


-3.2 


-8.8% 


School H Control 


41.1 


45.5 


44.7 


47.3 


46.1 


5.0 


12.3% 


51.9 


5.8 


12.6% 


School I 








58.8 


59.4 






52.1 


-7.3 


-12.3% 


School I Control 


50.3 


38.2 


58.2 


43.7 


61.2 


10.9 


21.7% 


72.0 


10.8 


17.6% 


Eighth-Grade English 






















School H 


36.9 


42.3 


35.8 


53.8 


37.4 


0.5 


1.4% 


28.3 


-9.2 


-24.5% 


School H Control 


35.6 


32.9 


39.8 


46.1 


41.0 


5.4 


15.2% 


37.2 


-3.8 


-9.2% 


School I 








42.4 


50.6 






42.7 


-7.9 


-15.7% 


School I Control 


50.8 


39.8 


46.6 


48.7 


66.7 


15.9 


31.3% 


67.1 


0.5 


0.7% 


Eighth-Grade Math 






















School H 


17.0 


23.6 


16.4 


26.0 


21.6 


4.7 


27.4% 


31.6 


10.0 


46.0% 


School H Control 


15.7 


22.3 


18.6 


25.5 


26.5 


10.8 


68.6% 


49.7 


23.3 


87.9% 


School I 








21.3 


35.2 






52.7 


17.5 


49.8% 


School I Control 


28.1 


25.3 


26.8 


39.6 


66.3 


38.1 


135.6% 


78.9 


12.7 


19.2% 


Eighth-Grade History 






















School H 


5.6 


9.3 


13.4 


19.7 


18.7 


13.1 


235.9% 


25.0 


6.3 


34.0% 


School H Control 


6.8 


4.0 


7.6 


5.0 


22.6 


15.8 


231.9% 


40.1 


17.5 


77.5% 
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Table 14 Changes in Percentages of Students Passing SOLs for Eighth-Grade Subjects (continued) 





Baseline 


Intervention 




1998 


1999 


2000 


2001 


2002 


1998-2002 

change 


1998-2002 % 
change 


2003 


2002-2003 

change 


2002-2003 
% change 


School I 








31.7 


25.1 






57.0 


31.9 


127.0% 


School I Control 


8.9 


13.4 


18.7 


31.8 


55.8 


46.9 


526.5% 


93.2 


37.5 


67.2% 


Eighth-Grade Science 






















School H 


32.2 


39.1 


42.9 


53.8 


44.0 


11.8 


36.7% 


38.9 


-5.1 


-11.6% 


School H Control 


30.3 


39.2 


39.5 


44.9 


53.1 


22.8 


75.1% 


53.3 


0.2 


0.5% 


School I 








51.5 


56.7 






47.3 


-9.4 


-16.6% 


School I Control 


50.3 


54.7 


62.4 


59.1 


81.8 


31.5 


62.7% 


73.6 


-8.2 


-10.0% 



Table 14 shows the following: 

• decreases in the percentages of Petersburg’s eighth-grade 
students who passed the SOL tests in writing, English, and 
science SOLs during the Model IV Intervention year 

• increases in the percentages of Richmond eighth-grade stu- 
dents who passed the writing SOL test; however, this varies 
for the School H control school, which also shows a de- 
crease in pass rate percentages for English, and for com- 
parison School I, which shows a percentage decrease for 
students who passed science SOLs during the Intervention 
year 

• gains in pass rate percentages for the Petersburg eighth- 
grade math and history SOLs, although Petersburg students 
in School H show smaller percentages in passing SOLs in 
math and history than do students in the Richmond com- 
parison Schools H. This situation is reversed for Petersburg 
School I and Richmond School I control. 

Table 15 shows changes in SOL pass rates for high school tests in 
writing, English, Algebra I, geometry, Algebra II, U.S. History, 
World History I, World History II, Earth science, biology, and 
chemistry for Petersburg schools for the five years prior to the in- 
tervention (1998 to 2002) and the Model IV Intervention year 
(2003). Comparable SOL data for the control schools are also in- 
dicated. 
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Table 15 Changes in Percentages of Students Passing SOLs for High School Subjects 





Baseline 


Intervention 


1998 


1999 


2000 


2001 


2002 


1998- 

2002 

change 


1998-2002 % 
change 


2003 


2002- 

2003 

change 


2002- 
2003 % 
change 


Writing 






















School J 


53.6 


67.7 


81.1 


79.2 


78.5 


24.9 


46.5% 


87.9 


9.4 


12.0% 


School J Con- 
trol 


44.8 


53.4 


50.0 


55.8 


65.0 


20.2 


45.0% 


82.7 


17.8 


27.3% 


English 






















School J 


56.5 


56.0 


63.6 


74.5 


81.5 


25.0 


44.3% 


89.1 


7.5 


9.2% 


School J Con- 
trol 


59.4 


41.4 


41.8 


65.4 


78.0 


18.6 


31.3% 


91.4 


13.5 


17.3% 


Algebra 






















School J 


6.6 


1.6 


8.8 


8.6 


17.8 


11.2 


171.3% 


44.7 


26.9 


151.0% 


School J Con- 
trol 


1.1 


3.4 


2.6 


7.2 


24.0 


22.9 


2156.1% 


50.8 


26.8 


111.5% 


Geometry 






















School J 


7.3 


19.3 


20.8 


25.0 


30.1 


22.8 


311.0% 


37.8 


7.7 


25.5% 


School J Con- 
trol 


10.3 


4.8 


8.8 


14.2 


32.9 


22.6 


218.7% 


50.0 


17.1 


52.0% 


Algebra II 






















School J 


3.6 


2.3 


8.5 


14.6 


16.9 


13.4 


377.1% 


44.4 


27.5 


162.4% 


School J Con- 
trol 


1.4 


1.3 


1.0 


12.0 


27.6 


26.1 


1801.9% 


34.5 


7.0 


25.3% 


U. S. History 






















School J 


11.9 


7.6 


6.6 


15.7 


29.2 


17.4 


146.4% 


48.2 


19.0 


65.1% 


School J Con- 
trol 


3.2 


1.3 


0.0 


10.4 


27.4 


24.2 


753.9% 


34.9 


7.6 


27.7% 


World History 
I 






















School J 






33.2 


37.8 


48.3 






50.2 






School J Con- 
trol 










66.7 












World History 
II 






















School J 


11.5 


11.9 


80.0 


18.0 


43.2 


31.7 


275.8% 


52.7 


9.5 


22.0% 


School J Con- 
trol 


3.8 


9.9 


4.2 


83.3 


39.5 


35.8 


953.5% 


38.3 


-1.2 


-3.1% 


Earth Science 






















School J 


25.3 


22.6 


30.2 


32.7 


23.3 


-2.0 


-8.0% 


48.9 


25.6 


110.0% 


School J Con- 
trol 


10.8 


20.0 


21.4 


25.0 


38.6 


27.8 


257.9% 


53.2 


14.7 


38.0% 
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Table 15 Changes in Percentages of Students Passing SOLs for High School Subjects (continued) 





Baseline 


Intervention 




1998 


1999 


2000 


2001 


2002 


1998- 

2002 

change 


1998-2002 % 
change 


2003 


2002- 

2003 

change 


2002- 
2003 % 
change 


Biology 






















School J 


37.7 


49.1 


54.4 


42.5 


54.2 


16.5 


43.9% 


46.8 


-7.4 


-13.6% 


School J Con- 
trol 


29.8 


44.1 


39.4 


45.3 


60.3 


30.4 


102.0% 


58.5 


-1.8 


-3.0% 


Chemistry 






















School J 


17.2 


12.3 


16.6 


24.2 


15.7 


-1.5 


-9.0% 


40.9 


25.3 


161.3% 


School J Con- 
trol 


9.1 


7.6 


7.4 


21.2 


17.6 


8.6 


94.1% 


32.8 


15.1 


85.8% 

























Table 15 shows increased percentages of students passing all high 
school SOL tests during the Model IV Intervention period except 
for the biology SOLs. For the most part, the percentages of stu- 
dents passing the SOLs appears to be greater for the Petersburg 
high school students than for the Richmond comparison schools 
except in writing, English, and geometry. 

Appendix 2 provides a series of charts to help the reader visualize 
trends in the SOL scores and to assist in detennining the effective- 
ness to date of the intervention program. The data used in these 
charts represent an average of the scores for each school district, 
and for each SOL test and year combination. 

An example is presented below in Figure 1, which displays the av- 
erage SOL pass rates for all seven elementary schools in the Pe- 
tersburg school system and for their comparison schools in the 
Richmond Division for the third-grade science SOL test. A least- 
squares linear trend (for the entire period from 1998 to 2003) is 
plotted for the Richmond comparison schools. For the Petersburg 
schools, a least-squares linear trend is plotted for the pre- 
intervention period (1998 to 2002) and for the intervention year 
2003 from the year prior (2002). 
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Eighth-Grade Science SOL Pass Rates and Trend Lines 




Year 



Figure 1. Third-grade science SOL test trends for Petersburg and control schools. 



The trend lines in Figure 1 show that, for the Richmond control 
schools, there has been a steady improvement in student pass rate 
averages for the third-grade science SOL test. The coefficient of 
determination (i.e., R-square statistic) for the Richmond schools’ 
trend line is 0.9, suggesting a high degree of correlation between 
the actual scores and the linear trend line. However, because we 
are not using these trends to forecast scores but instead to visualize 
trends — pre-intervention and during intervention — we concentrate 
on the relative slopes of the trends and the extent to which the Pe- 
tersburg schools exhibit a marked improvement in the scores dur- 
ing the intervention year. Accordingly, we observe that for this 
SOL test (third-grade science), the Petersburg schools have dem- 
onstrated a marked improvement in the pass rates for this SOL test, 
and that the trend in improvements is markedly higher during the 
intervention year than during the pre-intervention period (as dem- 
onstrated by the increased slope of the intervention period trend 
line for Petersburg schools). 

Charts for each of the SOL subject tests averaged for grades (ex- 
cept high school) are presented in Appendix 2. In summary, these 
charts demonstrate the following: 

• Third-grade SOL tests: The difference in achievement be- 
tween Petersburg schools and Richmond control schools 
appears to be lessening, with Petersburg schools making 
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larger gains and, hence, approximating the achievement of 
the Richmond controls. 

• Fifth-grade SOL tests: Results are mixed and should be ex- 
amined individually. 

• Eighth-grade SOL tests: Math and history pass rates for Pe- 
tersburg schools were generally lower than the control 
group during the pre-intervention period; however, notable 
gains have been made and the differences between Peters- 
burg and the Richmond control are becoming smaller. 

• High school SOL tests: Petersburg pass rates were gener- 
ally higher than the Richmond control pass rates during the 
pre-intervention period. Both high schools (Petersburg and 
the Richmond control) made gains in all subject areas. Pe- 
tersburg achievement is not increasing at a greater rate than 
Richmond City, but neither is it losing ground. 
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Conclusions and recommendations 

Conclusions 

Role of the school improvement specialists 

The school improvement specialists are the facilitators of the 
PA+SS Model IV Intervention. In the first two years of the inter- 
vention, these same seven school improvement specialists have 
worked closely with school leaders, instructional personnel, and 
the central office to foster adoption of a meaningful SIP and the 
use of data to guide instructional decisions. The difficulties they 
encountered have included cultural obstacles, training and educa- 
tion deficiencies, and management challenges. The school im- 
provement specialists alone cannot be expected to cause, or take 
credit for, progress in school perfonnance: a team effort is required 
for school improvement efforts to succeed. Even then, sustaining 
the progress could be problematic. 

As some school-based school improvement specialists commented, 
this intervention may take a significant amount of time to become 
sustainable. One school improvement specialist projected that 
reaching the level of sustainability may take 5 to 7 years. The 
school improvement specialists discovered that, in addition to time 
to perform their substantive work, another year or so on the front 
end was required to gain the trust of the people with whom they 
were working. By the time of our interviews, the school improve- 
ment specialists had, for the most part, been integrated into the 
schools’ activities. School leaders praised the school improvement 
specialists for their hard work and contributions. Some were quick 
to point out, however, that the school improvement specialist was 
only part of a team that also included school leaders and faculty 
members. 

It was clear that serving as a school improvement specialist can be 
frustrating and difficult. Much depends on the personality match 
between the school improvement specialist and school leadership. 
One school improvement specialist, who worked at two different 
schools, was described very differently by the two principals: “Did 
not try to take over the role of principal” and “Can be overbear- 
ing.” Thus, the “chemistry” between the school improvement spe- 
cialist and the principal is vital to success. School improvement 
specialists should be chosen for their tact and ability to perform ef- 
fectively in an ill-defined role, as well as for their prior successes 
as educators. 
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Several of the school improvement specialists commented on the 
problem of sustainment. They were concerned that their accom- 
plishments would not outlive their tenure at the schools. Their 
fears seem to be well founded. Unless substantive improvements 
are made in teacher quality, teacher competence is likely to remain 
problematic. High teacher turnover rates, combined with the Pe- 
tersburg School System’s policy of allowing teachers to be on staff 
for up to three years before acquiring state certification have exac- 
erbated this problem . 14 Anecdotal evidence indicates that teachers 
who use that time to gain accreditation move on to higher-paying 
schools once they become certified. Paradoxically, so do those 
who do not become certified. In either case, because the turnover 
is high and certification can be waived for up to three years, the 
potential for Petersburg schools to suffer from lower levels of 
teacher competence remains. 

Given the pressures on the schools to improve, the school im- 
provement specialists were torn between “giving the staff a fish or 
teaching them how to fish.” This tension was explicitly stated in 
one school improvement specialist’s journal. Were the school im- 
provement specialists better employed in teaching the staff how to 
disaggregate test data or by doing it themselves? Depending on 
the time pressure, the school improvement specialists took both 
approaches. 

The presence and activities of the school improvement specialist in 
the central office are central to this intervention. This part of the 
Model IV Intervention also provides the most fodder for hope in 
districts with systemwide deficiencies. If the school improvement 
specialist program, with the support of the superintendent, is able 
to get the culture turned around so that principals are held account- 
able for school improvement, progress is likely to follow. School 
administrators and, consequently, teachers will be more likely to 
accept and welcome the help offered by the school-level school 
improvement specialists when the central office shows strong sup- 
port for the principals. With assistance from the school improve- 
ment specialists, school staffs can make their school improvement 
plans the mechanism for focused attention that leads to improved 
schools and better-educated students. 



14 Five of the 10 Petersburg schools are designated by the Virginia Department of Educa- 
tion as “hard-to-staff’ schools. 
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A further issue not directly addressed here is the relationship be- 
tween parents and schools. In a reversal of good practice, one of 
the Petersburg principals removed parent representatives from the 
school improvement team. Yet involving parents is a key ingredi- 
ent to school improvement. 15 

Merits of the school improvement plans 

All schools complied with rubric indicators in SIP 1 Content of 
Plan. Difficulties arose when trying to detennine whether schools 
successfully incorporated the rubric’s indicators for SIP 2 Imple- 
mentation and SIP 3 Results into their plans. Few schools had 
completed the SIP indicator of progress as of October 1, 2004. 
Given the rudimentary nature of the data, it was difficult to deter- 
mine whether schools’ SIPs are being implemented. Moreover, it 
appears as though detailed documentation about implementation is 
the exception rather than the rule. 

The rubric provides guidelines to school improvement committee 
members for the purpose of shaping their school’s SIP content and 
implementation strategies and specifying measurement of school 
achievement results. School principals, school improvement spe- 
cialists, and teachers are typically members on the school im- 
provement committee. The school improvement specialists believe 
that the SIP presents a good indicator of where the school is 
headed in improving student achievement; however, SIPs do not 
necessarily address the school’s progress toward the stated goals. 
School improvement specialists repeatedly stressed that, although 
there is a great deal of interest and focus by the school principals 
and the central office on completing the SIPs, very little use is 
made of the completed documents. Two of the school improve- 
ment specialists noted that, although the SIP is a worthwhile en- 
deavor, it is not a tool by which to measure school success. 
Insufficient data exist regarding the role of the SIP in relation to 
student achievement on the SOL tests; this is largely due to the 
lack of specificity in the SIP document. When applying the rubric 



See the following for reviews of the importance of parent involvement: 

Epstein, J. (1992). School arid family partnerships. Baltimore: Center on Families, 
Communities, Schools and Children’s Learning. 

Levine, D. U., & Lezotte, L. W. (1990). Unusually effective schools: A review and 
analysis of research and practice. Madison, WI: National Center for Effective Schools 
Research and Development. 
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to the schools’ SIPs, it becomes apparent that whereas principals 
understand the level of content to include in their SIP planning, 
they do not grasp that the SIPs should include evidence of results 
appropriate for an AR (i.e., records documentation and 
data/measures [benchmark data] relevant to areas of improvement 
to indicate progress). 

The school improvement specialists from Schools C, D, and H 
mentioned that SIPs were not used to determine the schools’ pro- 
gress toward full implementation of the rubric indicators. “Instead 
of using them [SIPs], they [principals] put them on shelves where 
they [SIPs] were traditionally not used.” 

The school improvement specialists referred to the short time 
frame during which the SIPs goals were to be implemented. Be- 
cause school improvement specialists assisted heavily in the devel- 
opment of the SIPs, they were able to speak to the difficulty 
schools had in implementing all of the SIP goals while trying to 
deal with the day-to-day issues of school operations. Also of con- 
cern is the time taken by the schools to develop and implement the 
SIPs. Although the SIPs were to be completed in the first year of 
the Model IV Intervention, it was difficult for many of the schools 
to meet this deadline due to the high turnover rate of professional 
staff and their inexperience with results-based planning. In gen- 
eral, the school improvement specialists believe that the SIPs im- 
proved in the second year of the initiative. The year two SIPs were 
more detailed, and one could assume that as the schools become 
more comfortable with the structure and utility of the SIPs, the re- 
lationship between the SIP and SOL scores will improve. The cen- 
tral office school improvement specialist points out that “the 
challenge is to simply make the SIP more meaningful.” 



School culture and capacity for improvement 

Dimensions of school culture and capacity for improvement show 
significant increases in school improvement for climate on 5 of 6 
CSIQ dimensions, but capacity for improvement decreased signifi- 
cantly on 3 of 8 MSCI dimensions. The CSIQ shows that per- 
ceived changes in school culture are readily apparent when 2002 
and 2004 data are compared. However, changes in capacity for 
improvement show that professional staffs are less satisfied in 
2004 than they were in 2003. On an intuitive level, this is to be 
expected. A more favorable school climate and culture might well 
be the harbingers of overall capacity for school improvement. 
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In the elementary schools, there are a few exceptions to this gen- 
eral finding. The CSIQ shows that Schools B, D, and G display 
the highest mean percentile scores for 2004 on the learning culture 
scale: 49.4, 50.3, and 49.6, respectively. Schools B, D, and G 
each show a consistently upward trend in mean percentile scores 
on the shared goals for learning in 2004: School B (50.0), School 
D (51.9), and School G (49.7). Schools B, D, and G show the 
highest mean percentile scores of all elementary schools on the 
purposeful student assessment scales during 2004 (51.0, 50.9, and 
49.2, respectively). These are interesting findings, particularly in 
tenns of School B where the school improvement specialist reports 
that the school experienced high staff turnover. Schools B, D, and 
G showed increasing mean scores in 2004 on three to four of the 
eight MSCI scales, albeit not necessarily the same scales. 

According to some scales on the CSIQ, School B is proceeding 
well and, according to its principal, School B is closer than other 
elementary schools to getting school accreditation. This is occur- 
ring in the face of School B’s SIP having been evaluated as one of 
the most poorly documented plans at the elementary school level. 
As of March 17, 2004, the SIP had not been revised because 
teacher input had not been received. However, the school im- 
provement specialist reports that staff members do not appear to 
value the SIP and/or test data. Also, the school improvement spe- 
cialist had sporadic problems both in connecting with the principal 
and with the instructional specialist not fully understanding her 
role. In general, the school improvement specialist’s impression at 
School B indicated that staff felt no urgency to get anything done. 

School E tends to show overall mean percentile scores on most 
CSIQ scales, hovering between the 30 and 39 percentile range. 
Compared to other schools, this range tends to be low and speaks 
to the issue of School E’s weak administrative team. 



Standards of Learning 

Notable changes in the percentages of students passing the SOLs 
are occurring throughout Petersburg Schools. In general, there are 
positive gains in the percentages of fifth-grade students passing 
SOLs in each subject during the Model IV Intervention period. 
However, there are exceptions to this finding for Schools A and C, 
which show decreased percentages of students passing the writing 
SOL, and for Schools B and C which show decreased percentages 
of students passing the English SOL. In addition, decreases in the 
percentages of students passing SOLs in math and science continue 
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to be a problem for School C. The School C school improvement 
specialist said that her biggest disappointment this past year was 
“the lack of progress School C students made on the SOLs.” The 
school improvement specialist believes School C teachers are 
“buying into the Model IV Intervention, but the students aren’t 
there yet.” She is hoping for larger gains in School C SOL test re- 
sults in 2005. All of the mean percentile scores on the CSIQ scales 
for school climate at School C have increased during the Model IV 
Intervention period. If increased school climate is an indicator of 
expected increases in capacity for school improvement, then per- 
haps this may foretell improvement in SOL test scores. 

At the eighth-grade level, Petersburg students show gains in the 
pass rates for math and history SOLs, although these gains appear 
to be smaller than those for the Richmond control schools. Part of 
the issue regarding lower percentage changes for students passing 
the SOL tests may relate to the frequency of administrative 
changes in School H, where the principal changed twice during the 
2003-2004 school year. Not only was the administration in flux, 
classroom management was an issue. To address this issue, the 
school improvement specialist reported that during the first 45 
days of the 2004-2005 school year (the third year of the study), 
teachers would need to become more engaged with the overall 
management of school classrooms and use the Harry Wong Class- 
room Management Style. 16 With stable school leadership and ap- 
propriate classroom management, the percentages of students in 
Schools H and I who passed the SOLs appears to be poised to in- 
crease in 2005. 

Notable positive changes occurred in the percentages of School J 
students passing the SOLs from 1998 to 2001, that is, prior to the 
Model IV Intervention. These increasing percentages are continu- 
ing to occur, with the exception of biology, during the Model IV 
Intervention period. However, the school improvement specialist 
reports that the benchmark-testing program was not implemented 
in School J. With the hope that benchmark testing will be imple- 
mented in School J, SOL test scores appear to be on track for im- 
provement. One may speculate about whether the school 
improvement specialist’s concern about the high level of mistrust 
between School J and the central office may have thwarted bench- 
mark testing; nevertheless, the high level of mistrust may stand in 
the way of general progress toward school improvement, and this 



1 6 http://www.glavac.com/harrywong.htm 
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bothers the school improvement specialist. Within School J, 
teachers responded favorably about the school’s culture on the 
AEL CSIQ. The school’s mean percentile scores on the AEL 
CSIQ scales show improvement from 2002 to 2004 even though 
the AEL MSCI mean scores decrease somewhat on all AEL MSCI 
scales in 2004 from 2003. This finding may be indicative of the 
high staff turnover mentioned by the school improvement special- 
ist. 

To provide an alternative method of examining the achievement 
data, a least-squares linear trend (for the entire period from 1998 to 
2003) was plotted for the Richmond control schools. For the Pe- 
tersburg schools, a least-squares linear trend is plotted for the pre- 
intervention period (1998 to 2002) and for the intervention period 
(2002 to 2003). Petersburg schools are making gains in student 
achievement, often approximating or even exceeding the rate of 
gain of the matched Richmond control schools. 

With the high stakes involved in standardized testing under No 
Child Left Behind, all schools might have a tendency to improve 
regardless of strategic interventions. In fact, both the Petersburg 
schools and their matched control group schools are making gains. 
Given that, at this time, only one post-intervention year could be 
compared to the pre-intervention achievement data, more time is 
needed to see if the Model IV Intervention will have the intended 
effects on student achievement. 

Recommendations 



The school instructional specialist could play a key role in the sus- 
tainability of improvement efforts over time. 17 Ideally, this indi- 
vidual will provide continuity in data expertise and be available 
both to provide instruction in data analysis and to perform data 
analysis as needed. Institutionalizing this position, after ensuring 
that the instructional specialists are well grounded in data analysis, 
would go a long way toward addressing the problem of sustainabil- 
ity in individual schools. Therefore, the school improvement spe- 
cialists should continue to train instructional specialists in data 
analysis throughout the three-year Model IV Intervention. 



17 

The school instructional specialist should not be confused with the school improve- 
ment specialist. 
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Because the role of the parent is critical to student success, all 
principals should be required to include parent representatives on 
each school improvement committee to help develop each school’s 
SIP. 

The Model IV Intervention innovation of assigning a senior school 
improvement specialist to the central office appears to be having 
some positive effects in the Petersburg Division, based on inter- 
view, survey, and achievement data. Therefore, the intervention 
team should work with the Division to ensure that the specialist's 
responsibilities are picked up, either by creating a pennanent posi- 
tion or by assigning these responsibilities to another central office 
staff member who has the necessary qualifications. 

An effort should be made to reinforce and encourage the use of the 
SIP as a tool for the school’s administration to gauge school pro- 
gress toward defined goals. The SIP should be “a living docu- 
ment” (a theme echoed throughout the school improvement 
specialist interviews), and one should be created and monitored for 
the Division and for each school. This would help the central of- 
fice be more involved in supporting principals, building school ca- 
pacity, and maintaining collaborative relationships with school 
leaders. 

School leaders should be required to focus on specific strategies of 
the SIP during a set time frame. Instead of requiring schools to 
implement the entire SIP all at once, the SIP might better be ad- 
ministered in an easily managed order, predetermined either at the 
central office level or at the individual school level and supported 
by the central office. 

Because an increasingly favorable school culture might well be a 
harbinger of the capacity for improvement, it will be important to 
measure staff perceptions in spring 2005 to determine whether the 
schools continue to develop cultures of continuous improvement 
and capacity for improvement. 

Instructional specialists need to place more emphasis on building 
capacity among the professional teaching staff by stressing the 
alignment of curriculum with the SOLs and using the evidence- 
based practices available to them for this purpose. According to 
one district leader, the Petersburg school division’s most pressing 
need continues to be having in place “an effectively organized cur- 
riculum delivery system.” 
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Appendixes 




Appendix 1 - Rubric 



Al-l 




SCHOOL-LEVEL ACADEMIC REVIEW School Code: 

Form 4-SIP: SCHOOL IMPROVEMENT PLAN, Data Collection Summary Sheet Date ot Visit: 

Page 1 of 7 



Content Area(s) Reviewed: English: Mathematics; History; Science 





Sources of Evidence 


Significant Findings 

If indicator is cited as AOS or Al, or both, the team's 
Significant Findings must be provided below. 


AOS 


A! 


Do NOT reword the indicator in the report. Include a clarifying 


Self- 

Study 


Doc Int j 


Obs 


SIP 1 Content of Plan 

SIP 1.1 Basing the three-year school improvement plan on the 
results of previous academic reviews, as required by 
the Standards of Accreditation 
















SIP 1.2 Developing the three-year school improvement plan 
with the assistance of parents and teachers, as 
required by the Standards of Accreditation 
















SIP 1 .3 Using baseline data/measures relevant to areas for 
improvement to identify goals for school improvement 
planning 
















SIP 1.4 Establishing clear goals that relate to student 
achievement 














1 


SIP 1.5 Establishing yearly, measurable objectives or 
benchmarks that are linked to goals 










1 
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SCHOOL-LEVEL ACADEMIC REVIEW School Code: 

Form 4-SIP: SCHOOL IMPROVEMENT PLAN, Data Collection Summary Sheet Date of Visit: 

Page 2 ol 7 



Indicators 

Do NOT reword the indicator in the report. Include a dantying 
comment if the indicator, as written , does not "fit" 


Sources of Evidence 


Significant Findings 

If indicator is cited as AOS or Al. or both, the team's 
Significant Findings must be provided below. 


AOS 


Al 


Self- 

Study 


Doc 


Int 


Obc 


SIP 1.6 Describing the strategies to be implemented and the 
specific action steps to be taken to meet each 
objective 
















SIP 1 .7 Including data collection activities at regular, logical 
(not random) intervals throughout plan, as part of 
strategies/action steps 
















SIP 1.8 Including a system of monitoring student progress at 
regular, logical (not random) intervals 
















SIP 1 .9 Selecting achievement indicators that are appropriate 
to goals and objectives 














SIP 1.10 Identifying sources of evidence that are appropriate to 
strategies/action steps 
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SCHOOL-LEVEL ACADEMIC REVIEW 

Form 4-SIP: SCHOOL IMPROVEMENT PLAN, Data Collection Summary Sheet 

Page 3 of 7 



School Code: 
Date of Visit: 



Indicators 

Do NOT reword the indicator in the report. Include a clarifying 
comment il the indicator, as written, does not ‘tit ' 


Sources of Evidence 


Significant Findings 

If indicator is cited as AOS or Al. or both, the team's 
Significant Findings must be provided below. 


AOS 


Al 


Self- 

Study 


Doc 


Int 


Obs 


SIP 1.11 Identifying person(s) responsible for implementing 
strategies/action steps and collecting datatevidence 
















SIP 1.12 Establishing limelines over a three-year period and 
linking shorter timeframes to specific action 
steps/strategies 
















SIP 1.13 Including all nine components required by Section 
8VAC20-131-310.G of the Standards of Accreditation. 
(Refer to TA Document in AR Users Handbook or 
SOA for list of components.) 
















SIP 2 Implementation 

SIP 2.1 Focusing implementation on improved student 
achievement 
















SIP 2.2 Implementing strategies and action steps in the 
manner described in plan 
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SCHOOL-LEVEL ACADEMIC REVIEW 

Form 4-SIP: SCHOOL IMPROVEMENT PLAN, Data Collection Summary Sheet 

Page 4 of 7 



School Code: 
Date of Visit: 



Indicators 

Do NOT record the indicator m the report. Include a clarifying 
comment if the indicator, as written, does not 'Tit ", 


Sources of Evidence 


Significant Findings 

If indicator is cited as AOS or At, or both, the team 's 

Significant Findings must be provided below 


AOS 


Al 


Self* 

Study 


Doc 


Int 


Obs 


SIP 2.3 Documenting implementation ol strategies/action steps 
















SIP 2.4 Collecting and compiling data/evidence of the degree 
to which strategies and action steps are implemented 
as described in the plan 
















SIP 2.5 Monitoring the efforts of staff in carrying out their 
responsibilities 
















SIP 2.6 Meeting established timelines for completing 

strategies/action steps and collecting data/evidence 
















SIP 2.7 Establishing procedures for macro-to-micro analysis of 
data 
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SCHOOL-LEVEL ACADEMIC REVIEW School Code: 

Form 4-SIP: SCHOOL IMPROVEMENT PLAN, Data Collection Summary Sheet Date of Visit: 

Page 5 ot 7 



Indicators 

Do NOT reword the indicator in the report. Include a clarifying 
comment if the indicator, as written, does not HF. 


Sources of Evidence 


Significant Findings 

If indicator is cited as AOS or Al. or both, the team s 
Significant Findings must be provided below. 


AOS 


Al 


Self- 

Study 


Doc 


Int 


Obs 


SIP 3 Results 

SIP 3.1 Establishing a system for monitoring and adjusting the 
school's improvement plan 
















SIP 3.2 Analyzing data/evidence to determine the degree to 
which strategiestections steps are implemented as 
intended 
















SIP 3.3 Systematically monitoring student achievement at 
regular intervals throughout the year to determine 
effectiveness of improvement initiatives 
















SIP 3.4 Using data to determine the impact of improvement 
initiatives on student learning 

















SIP 3.5 Modifying goals and/or objectives based upon the 
analyses of date/evidence 
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SCHOOL-LEVEL ACADEMIC REVIEW School Code: - 

Form 4-SIP: SCHOOL IMPROVEMENT PLAN, Data Collection Summary Sheet Date of Visit: 

Page 6 of 7 



mmmmm 


v. vr Sources of Evidence; .Jv Sianiftcatiit ftitdlks ‘f;- 

Sfllf- I ^ ; liu 1 t: lfMi^risdledasAI^M,^niie!eam'f 
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AOS 

- 


Al 


SIP 3.6 Modifying less successful strategies and adding new 
strategies, as needed, to promote continued 
improvement 
















SIP 3.7 Reaching established student achievement 
benchmarks and/or objectives 
















SIP 3.8 Communicating the status of implementation and the 
results to stakeholders 
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SCHOOL-LEVEL ACADEMIC REVIEW 

Form 4-SIP: SCHOOL IMPROVEMENT PLAN, Data Collection Summary Sheet 

Page 7 of 7 

NOTES: 



Foilow-Uo Questions/Answers 


Additional Relevant Information: 






Clarification of sources of evidence (as needed): 


Preliminary Essential Actions: 


Summary Statements (as needed): 



School Code: 
Date of Visit: 
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Appendix 2 - Trend lines for average percent- 
ages of students passing SOL tests 



A2-1 




Average Percent of Students Passing SOL Average Percent of Students Passing SOL 



Third-Grade English SOL Pass Rates and Trend Lines 




Year 



Third-Grade Math SOL Pass Rates and Trend Lines 




Year 



A2-2 



Third-Grade History SOL Pass Rates and Trend Lines 




Third-Grade Science SOL Pass Rates and Trend Lines 




Year 



A2-3 



. „ . * Average Percent of Students Passing 

Average Percent of Students Passing SOL 



Fifth-Grade Writing SOL Pass Rates and Trend Lines 




Year 



Fifth-Grade English SOL Pass Rates and Trend Lines 




Year 



A2-4 



Average Percent of Students Passing SOL Average Percent of Students Passing SOL 



Fifth-Grade Math SOL Pass Rates and Trend Lines 




Year 



Fifth-Grade History SOL Pass Rates and Trend Lines 




A2-5 



Average Percent of Students Passing SOL 



Fifth-Grade Science SOL Pass Rates and Trend Lines 




Year 



Eighth-Grade Writing SOL Pass Rates and Trend Lines 




1997 1998 1999 2000 2001 2002 2003 2004 



Year 



A2-6 



Eighth-Grade English SOL Pass Rates and Trend Lines 




Year 



Eighth-Grade Math SOL Pass Rates and Trend Lines 




Year 



A2-7 



Average Percent of Students Passing SOL Average Percent of Students Passing SOL 



Eighth-Grade History SOL Pass Rates and Trend Lines 




Year 



Eighth-Grade Science SOL Pass Rates and Trend Lines 




Year 



A2-8 



High School Writing End-of-Course Pass Rates and Trend Lines 




Year 



High School English End-of-Course Pass Rates and Trend Lines 




Year 



A2-9 



High School Algebra I End-of-Course Pass Rates and Trend Lines 




High School Geometry End-of-Course Pass Rates and Trend Lines 




A2-10 



High School U.S. History End-of-Course Pass Rates and Trend Lines 




Year 



High School Algebra II End-of-Course Pass Rates and Trend Lines 




A2-11 



Percentages of Students Passing Exam Percentages of Students Passing Exam 



High School World History I End-of-Course Pass Rates and Trend Lines 




Year 



High School World History II End-of-Course Pass Rates and Trend Lines 




Year 



A2-12 



Percentages of Students Passing Exam Percentages of Students Passing Exam 



High School Geography End-of-Course Pass Rates and Trend Lines 
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High School Earth Science End-of-Course Pass Rates and Trend Lines 




Year 



A2-1 



Percentages of Students Passing Exam Percentages of Students Passing Exam 



High School Biology End-of-Course Pass Rates and Trend Lines 




Year 



High School Chemistry End-of-Course Pass Rates and Trend Lines 




Year 



A2-2 



